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Abstract 


This report reviews the literature on noncognitive and other background predictors (e.g., 
personality, attitudes, and interests) as it pertains to graduate education. The first section reviews 
measures typically used in studies of graduate school outcomes, such as attrition and time to 
degree. A review of qualities faculty members and administrators say they desire and cultivate in 
graduate programs is conducted. There appears to be a divergence between the qualities faculty 
members say are important and the measures researchers typically use in validity studies. 

The second section reviews three categories of noncognitive variables that might predict 
outcomes (general personality factors, quasi-cognitive factors, and attitudinal factors) and the 
definitions, measures, correlates, and the validity of those measures. The role of background 
factors, both environmental and group, is part of this review. 

Key words: noncognitive, personality, admissions, GRE 8 ’, selection, assessment, new constructs 
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Executive Summary 

Many GRE® studies conducted in the past several years have asked faculty members 
about the qualities that are important to success in graduate school. A consistent finding is that 
graduate faculty members believe that noncognitive factors, such as motivation, creativity, 
personality, interests, and attitude, ought to be considered in graduate admissions (e.g., Briel et 
ah, 2000). Many faculty members believe that including such factors would increase both 
fairness and the validity of the current admissions process. At the same time there is a lack of 
knowledge about what the scientific literature says on what the noncognitive factors are, how 
important they are, how they could best be measured, and on whether it might be practical to use 
them in admissions decisions. The purpose of this review is to address these questions. This 
report reviews various noncognitive factors and definitions, measures, correlates, and the validity 
of those measures. 

Graduate School Outcomes and Their Determinants 

A notional path model illustrates the relationships between noncognitive predictor factors 
and graduate school outcome factors. Graduate school outcome factors are divided into 
traditional measures of convenience and performance factors. Traditional measures are attrition, 
time-to-degree, and grade point average. Performance factors are domain proficiency, general 
proficiency, communication, effort, discipline, teamwork, leadership, and management. The 
noncognitive predictor factors are divided into three main categories: personality (e.g., 
extroversion), quasi-cognitive (e.g., metacognition), and motivation factors (e.g., self-efficacy). 
In addition, there are two background categories assumed to affect the noncognitive factors: 
environmental (e.g., mentor support) and group factors (gender and ethnicity). The review is 
organized according to this model. 

Personality Factors 

A consensus has emerged within the field that there are five major personality factors: 
extroversion, emotional stability, agreeableness, conscientiousness, and openness. Other 
personality factors appearing in the literature can be seen as combinations or facets (narrower 
subfactors) of these. 

The advantages of including personality in graduate admissions review are that doing so 
would broaden the qualities considered and increase ethnic diversity. Faculty members say 



personality is important. The problem is that personality is measured through self-reports and is 
therefore fakable. The literature suggests techniques to control faking (e.g., adjust scores for 
social desirability bias, use forced choice, warn examinees not to fake, use subtle items) but these 
have not been tried in large volume, high stakes settings. The techniques would likely fail under 
such circumstances because coaching could be developed to overcome them. 

There are two ways personality could be introduced into graduate admissions. One is 
through the use of a perfonnance (ability) test for measuring personality, which could be 
immune to faking. One new method that may have promise is the conditional reasoning 
approach, which involves items that look like reasoning items, but contain more than one correct 
answer, the choice of which is thought to reflect personality. This approach is too new to have 
been evaluated sufficiently. Another way is through the use of others’ rating of personality— 
advisors, professors, and other members of the college community who typically write letters of 
recommendation for students now. 

Quasi-Cognitive Factors 

These are factors that are often considered somewhere in between cognitive and 
noncognitive factors. They may be measured with perfonnance tests, or ability tests, but they 
also reflect affective qualities. The review considers four such factors: creativity, emotional 
intelligence, metacognition and confidence, and cognitive style. 

Creativity 

Creativity typically ranks high on the list of qualities faculty say are important to success 
in graduate school. However, people disagree on what exactly it is, and on how it ought to be 
measured. Self-reports tend to be obvious, and easily faked. Performance measures, such as 
fluency tests (e.g., “How many ways can you use a brick?”) tend not to predict important criteria, 
independently of general cognitive ability. Creativity has been assessed by others’ judgments, as 
overall ratings of the person, or as ratings of particular creative products (e.g., essays). Whether 
such ratings are independent of other quality judgments has not been established, but such an 
approach illustrates that creativity could be and has been used as an outcome (criterion) variable 
as well as a predictor. 



Emotional Intelligence 

Emotional intelligence has received considerable attention in the popular media and some 
attention in scientific literature. Measures fall into two categories: self-reports and performance 
tests. Self-reports yield scores that duplicate personality measures, and they have the same 
problems as personality measures—they are fakable. Perfonnance tests are possibly a different 
story. They have not been studied sufficiently, but they offer the promise of not being fakable. 
However, little is known about their validity. 

Confidence, Metacognition 

These are measures that assess whether examinees can accurately predict whether they 
know the correct answer to a test item. There has been considerable research suggesting that 
one’s ability to predict is independent of one’s ability per se. However, it is not clear how this 
could be used in applied admissions contexts. A possible application might be to teach more 
accurate self-monitoring, which could be considered both a study and test-taking skill. 

Cognitive Style 

Considerable research has been conducted on cognitive styles such as field-dependence, 
but there have been problems in discriminant validity (e.g., overlap with cognitive ability or 
personality). Also, for some cognitive styles, those based on self-reports, fakability probably 
precludes their use in admissions. 

Attitudinal Factors 

Attitudinal factors influence choice of activities, goals, strategies, effort, and persistence. 
For this review, attitudinal factors are self-concept, self-efficacy, motivation, attributions, and 
interests, and social attitudes. Attitudes are often domain specific. For example, being motivated 
in one domain, such as academics, may be largely independent from being motivated in another, 
such as athletics. Attitudinal factors are thought to be particularly important in understanding 
students traditionally underrepresented in graduate programs. 

Self-Concept and Domain Identification 

This refers to the way people characteristically think about themselves in a domain. The 
focus here is on academic self-concept, and identification with the academic domain. Self-views 
and values determine the importance one puts on achieving and on the behaviors one engages in. 
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Many factors influence self-concept, including background experiences, race, and gender. Self- 
concept is typically measured with self-assessments, but a measure called the implicit association 
test could possibly be used as a self-concept measure. 

Self-Efficacy 

This widely researched construct refers to one’s belief in one’s ability to achieve success 
in a particular area. Self-efficacy is important in selecting goals, adopting strategies, task 
persistence, and the effort put into a task. High self-efficacy is associated with a wide range of 
positive academic outcomes. Self-efficacy can be influenced by numerous causes, such as 
domain mastery, faculty support, personality, and interests. There are many programs designed 
to increase self-efficacy in a variety of areas, in addition to academic ones. 

Motivation 

The term motivation has been used in many different ways over the years, which makes it 
difficult to pin down as a psychological construct. Much of the motivation research literature 
falls into other categories (e.g., self-efficacy), but there are a few concepts associated with the 
topic of motivation per se. One is the distinction between extrinsic and intrinsic motivation, with 
intrinsic motivation widely believed to be related to higher outcomes, a belief not supported by 
recent meta-analytic research. Another is the distinction between a performance- and a learning- 
goal orientation, with the success of the orientation depending on one’s ability (performance 
goals work well when one is proficient; otherwise, learning goals work better). 

Interests 

A dominant framework for studying interests is Holland’s (1959, 1973) six dimensions 
scheme, which identifies realistic, artistic, investigative, social, enterprising, and conventional 
interests. Numerous studies have shown li nk s between interests and academic outcomes. There 
are gender differences in interests (men higher in realistic; women higher in social and artistic), 
which undoubtedly accounts for some of the differences in what graduate fields of study men 
and women pursue. 


v 



Attributions 

We habitually attribute cause for successes and failures to others, stable or unstable 
circumstances, or ourselves and to forces under our control or not under our control. Adaptive 
attribution styles (e.g., attributing failures to ourselves, but also to forces we can control, and that 
are changeable) leads to greater persistence and intensity in performing tasks, and therefore may 
be of considerable value in higher education achievement. 

Social Attitudes and Values 

There have been several proposals for the structure of beliefs, attitudes, and values, such 
as individualism, equality, and religiosity. These have primarily been studied in cross-cultural 
contexts, comparing countries with each other, and relating these to national indicators such as 
GNP and literacy. However, it may be that social attitudes and values are also useful to study at 
the individual level, as they pertain to success in higher education. 

Environmental Influences on Graduate School Outcomes 

This section is a discussion of variables other than personality and motivation that may 
affect performance in graduate school. 

Mentor and Social Support 

Mentor and social group support affect persistence to degree, as well as other less 
tangible outcomes such emotional well-being. Informal contact with one’s mentor, and mentor 
qualities such as interest in the student, accessibility, integrity, reliability, and communication 
skills are important for various graduate school student outcomes. 

Prejudice and Institutional Integration 

Encountering negative stereotypes about the intelligence or abilities of one’s gender, 
ethnic, or age group may have negative effects on attitudes and school and test performance. 
There have been numerous demonstrations in the laboratory of such effects, but few in actual 
higher-education settings. One way prejudice may impair school perfonnance is by inhibiting 
integration into the university environment, leading to attrition. There have been various 
programs and policies designed to increase integration, particularly for female and Black 
students. 
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Financial Support 

Financial support reduces attrition and leads to better dissertations, but it is not equally 
available to all. There is controversy over whether fellowships or assistantships are more 
effective, with two major research studies coming to different conclusions. 

Prior Accomplishments 

Prior accomplishments, as reflected in transcripts, resumes, and standardized surveys, 
predict graduate school accomplishments, and various undergraduate criteria. Although not 
widely researched, self-assessed standardized accomplishments measures, particularly when 
verifiable, have proven useful and may warrant further evaluation. 

Group Factors 

Black, Hispanic, and female students have lower participation, PhD candidacy, and 
graduation rates than White and male students. Issues are what factors are responsible and what 
might be done about it. 

Ethnicity 

Several explanations have been invoked to account for lower standardized test scores and 
higher attrition and lower perfonnance in school of Black and Hispanic students. One is 
stereotype threat, which causes disidentification from the academic domain. Another is systemic, 
which affects identify development. Academic preparation is lower for Black and Hispanic 
students, as indicated by high school science and mathematics courses, and the selectivity of 
colleges attended. To some degree, faculty support, financial aid, and institution policies may 
combat these effects, as might consideration of a broader range of factors in admissions 
decisions. 

Gender 

The gender gap is closing, but differences remain in science, mathematics, and 
engineering, and in standardized test scores. Some of these differences reflect different interests 
between genders. Differences in academic preparation are rapidly closing, particularly in high 
school. Some of the difficulties women experience in mathematics and science may be reduced 
by considering a broader range of factors in admissions, coupled with institutional and faculty 


vii 



support. But the downside of diminished peer respect for abilities of those in support and 
preferential programs has been noted. 

Summary 

The purpose of this review was to consider the role played by noncognitive factors in 
graduate school, and how such factors could be used, particularly in graduate admissions. 

Several recent studies sponsored by the GRE R program have surveyed faculty to identify the 
qualities they value in admissions and as program outcomes. The information is summarized 
here and the research literature about those qualities—what they are, how they are measured, 
what they are related to, and whether they predict educational success is explored. Based on the 
findings, there are specific opportunities in noncognitive assessment for graduate education 
available to pursue now. We exclude from consideration the simple adoption of personality and 
motivation self-assessments due to the threat of coaching and fakeability. But there are other 
ways noncognitive variables might be used. In admissions, one can imagine using noncognitive 
variables in the creation of a guide for writing or interpreting letters of recommendation. Current 
findings suggest dimensions, items, and approaches for such an endeavor. There also may be 
some performance-based personality, attitude, and emotional intelligence measures that could 
prove useful in admissions applications, and that merit further research. Outside of admissions, a 
possible application is guidance where noncognitive information could be used to help students 
find compatible graduate study programs. A third possibility is using noncognitive constructs as 
graduate school outcome measures, which could support an institution’s efforts to both analyze 
and promote its programs. 
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Overview 

Is there a role for noncognitive variables—personality, motivation, self-efficacy, life 
experiences and accomplishments—in graduate admissions? Many in the graduate education 
community believe so. Graduate admissions staff interviewed in the recently concluded Horizons 
project (Briel et ah, 2000) frequently mentioned the need for noncognitive indicators to augment 
the cognitive measures of the GRE. A recent meta-analysis of the validity of the GRE also 
concluded that improved validity of graduate student selection would only come about by 
including noncognitive variables such as personality and interests (Kuncel, Hezlett, & Ones, 

2001). Outside of higher education—in industry and the military—there is a widespread belief that 
a major breakthrough in further progress on predicting success in education, training, and 
performance settings will involve considering noncognitive factors (Hough, 2001; Schmidt, 1994). 

Interest in the role of factors other than strictly cognitive ones in higher education is long 
standing. Over the years there have been many conceptual and empirical investigations of 
noncognitive factors. With few exceptions this interest has not led to large-scale development of 
new assessments that might be used in admissions or guidance. 

What is different now? One difference is that after an initial flurry of activity related to new 
possibilities for incorporating cognitive psychology in mental testing (e.g., Sternberg, 1977), a 
failure to find factors other than general cognitive ability to predict learning outcomes led to a 
period of stagnation in the field. Many researchers concluded that advances in our ability to predict 
learning outcomes would have to come from outside the cognitive realm (Schmidt, 1994). A 
second factor is that for many years, a predominant view in psychology was that situational factors 
were more important than personal ones in governing behavior (Mischel, 1977). The concept of 
personality itself was questioned, and by some, even considered illusory. The rediscovery of the 
five factor model (Goldberg, 1992), and demonstrations of its robustness across cultures and 
language groups, opened the door to renewed theorizing about personality, and made the 1990s a 
kind of golden decade of personality research. A third factor might have been the publication in 
1995 of the enormously successful book, Emotional Intelligence, which sold over 5 million copies, 
appeared as a cover story in Time and Newsweek, and whose author, Daniel Goleman, was a highly 
sought after guest on popular television and radio shows (Goleman, 1995). The book’s message 
was that there is more to success than intelligence. And finally, researchers from various circles 
began expressing the idea that it was important to consider a wider range of criteria. For example, 
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Sternberg (1986) argued that there were several dimensions of success, including creative and 
practical as well as analytic intelligence. Campbell, McCloy, Oppler, and Sager (1993) provided 
evidence for multiple criterion perfonnance dimensions in military and organizational settings, a 
topic to which we return shortly. 

Purpose of This Review 

The purpose of this review is twofold. First, there is an exploration of definitions of 
graduate student success. The review considers what qualities graduate faculty look for in 
students during the admissions process and beyond. Second, a review of noncognitive factors 
that might affect success in graduate school is undertaken. Consideration of both how those 
factors are measured and how they relate to graduate school success follows. 

Organization of This Review 

The dividing factors are in three groups: background predictor factors (group and 
environmental factors), noncognitive predictor factors (personality, attitudinal, and “quasi- 
cognitive” factors), and outcome criterion factors (convenience and performance measures, and 
affective factors). Figure 1 illustrates a tentative model of the relationships among these. 

This report proposes that personality, attitude, and quasi-cognitive factors directly affect 
graduate school outcomes. Background factors affect graduate school outcomes indirectly, 
through attitude. There are also group differences in personality and the quasi-cognitive factors. 
The model is tentative, based not on a systematic meta-analysis, but on an initial understanding 
of how these factors interact. There are testable alternatives to this model, such as allowing direct 
effects of background factors on outcomes, or allowing paths between sub factors rather than 
groups of factors. Nevertheless there is benefit to this as a starting point. 

Graduate School Outcome Factors 

Graduate school outcomes that receive the most attention, in admissions offices and 
among researchers, are attrition, time to degree, and student quality. The latter is measured by 
faculty ratings, grade point average (first year and overall), and scores on comprehensive 
examinations. The reason for their prominence is partly convenience—colleges keep these sorts 
of records for a variety of purposes; it is easy to retrieve them and discuss them. But all have 
their problems. Grade point average, despite being the most widely used, lacks standardization 
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across schools, degree programs, and individual faculty members. Ratings similarly lack 
standardization. Attrition and time to degree are complex variables with many, varied 
determinants. For example, a student might drop out due to a family illness or a job opportunity. 
Are these the right measures of graduate school outcomes? How important are they to the 
graduate community? What factors does the graduate community value? 


Background factors 


Noncognitive factors 


Outcome factors 



Convenience measures 

Grade point average 

Exams 

Ratings 

Attrition 

Time to degree 

Productivity 

Performance factors 

Domain proficiency 

General proficiency 

Communication 

Effort 

Discipline 

Teamwork 

Leadership 

Management 

Affective factors 

Liking 

Interest 

Attitudinal factors 


Figure 1. Tentative model of the relationships among background predictor, 
noncognitive predictor, and outcome criterion factors. 
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Indicators of Graduate School Success 

There have been several formal studies of graduate school outcomes to address these 
issues in recent years, based on interviews conducted with faculty and administrators. 

Graduate school competencies project. Enright and Gitomer (1989) interviewed 15 
“noted scholars and mentors with direct experience in graduate teaching” (p. 2), with the goal of 
identifying competencies important for graduate school success. As a result of two sets of 
interviews (over the phone, and a follow-up group discussion), and a review of the literature on 
graduate school criteria, Enright and Gitomer identified “seven general competencies.” These, 
with definitions and examples, are listed in Table 1. Note that these are outcome success criteria 
in that they are competencies graduate faculty would like to see in their students. Some of these 
competencies might be knowable, to some extent, prior to an admissions decision. For example, 
undergraduates write reports, display creativity, develop logical arguments, go beyond the 
minimum, design experiments, display social skills, and function independently. But the 
important point is that they are the competencies graduate faculty accept as outcomes and 
indicators of success in graduate school. 

The first two competencies, and maybe the first three, all seem to be essentially 
cognitive, and closest to what the GRE currently measures. The next one—communication— 
starts moving away from the current GRE (although it may be reflected in the new essay 
assessment). The remaining others are more distinct and noncognitive in nature. 

Horizons project. More recently, as a follow-up to the Horizons project (Briel et al., 
2000), and as part of a larger study on GRE validity, researchers conducted lengthy telephone 
interviews with 16 faculty members and five deans from five institutions, including both doctoral 
and masters institutions, and including one Hispanic-serving institution and one historically Black 
college or university (HBCU) (Walpole, Burton, Kanyi, & Jackenthal, 2001). A purpose of the 
interview was to query respondents for definitions of successful graduate students. In open-ended 
discussions, a total of 36 attributes were mentioned by the 21 interviewees, and the authors scored 
the attributes by characterizing how they were talked about as either enthusiastic (3 points), 
moderate (2 points), or conditional (1 point). No mention of any of these resulted in zero points 
awarded. Some variables were talked about in relation to admissions, others to outcomes, and 
some, to both. Table 2 presents the list of 21 attributes mentioned in relation to outcomes, sorted 
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by their average score (the total number of mention points divided by 21, the number of 
interviewees). An analysis of mentions in regards to admissions will be presented later. 

A noteworthy finding is the number of noncognitive attributes mentioned in Table 2. 
Some indeed are mentioned quite frequently—persistence (1.48), collegiality (1.38), and 
communication (1.14), were among the most frequently mentioned outcomes, even more 
frequently mentioned than mastery of the discipline (1.00) and ability to teach (.62). 


Table 1 


Enright and Gitomer: Seven Important General Competencies 


Competency 

Definition 

Examples 

Explanation 

Giving a reason or cause for 
some phenomenon or finding 

Produce alternative hypotheses, 
analogical reasoning, develop 
logical argument, defend ideas 

Synthesis 

Skills that facilitate the 
development of expert domain- 
knowledge structures 

Master content and skills in a field, 
function independently, 
manipulate knowledge creatively 

Creativity 

Produce unusual ideas, generate 
novel ideas 

Intellectual playfulness, 
rebelliousness—criticize, modify 
facts, theories 

Communication 

Ability to share ideas, 
knowledge, and insights with 
others 

Write reports, give presentations, 
discussions with others 

Motivation 

Commitment, involvement, 
interest in work 

Persistence, enthusiasm, 
excitement, go beyond minimum 

Planning 

(self-organization) 

Developing a procedure to 
reach a goal 

Design an experiment, organize a 
paper, make career decisions 

Professionalism 

(social) 

Skills in accommodating to 
social conditions in field 

Social skills, joining professional 
societies, informal discussions 
with faculty and students 
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Table 2 


Mean Importance Ratings of Outcomes of Open-Ended Discussions 


Variable 

Mean rating 

Research/scholarly experience (amount and quality) 

1.52 

Persistence/tenacity 

1.48 

Collegiality/making professional connections/networking 

1.38 

Communication (professional) 

1.14 

Mastery of discipline 

1.00 

ESL ability 

0.71 

Ability to teach 

0.62 

Independence 

0.43 

Creativity: ability to think “out of the box” 

0.43 

Enthusiasm 

0.43 

Ability to read and analyze research in the field 

0.43 

Values/character: integrity, fairness/openness/honesty, 
trustworthiness/consistency—personal and professional 

0.29 

Breadth of perspective 

0.29 

Open mindedness 

0.29 

Opportunism/resourcefulness 

0.29 

Critical thinking ability, logic, problem solving ability 

0.14 

Drive/commitment/motivation/zeal 

0.14 

Computer literacy/understand and manage technology 

0.14 

Skill in investigation 

0.14 

Ability to find the facts: information gathering 

0.14 

Professional posture 

0.14 


Note. Importance rating is the mean mention level of the variable across the 21 participants, where 
0 = not mentioned, 1 = conditional mention, 2 = moderate mention, 3 = enthusiastic mention (from 
Walpole et ah, 2001, Table 1); therefore higher ratings indicate greater endorsement. 
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A taxonomy of higher order performance components. It would be useful to categorize 
this list of attributes. A well-established scheme for classifying outcome or criterion variables is the 
taxonomy of higher order performance components (Campbell et ah, 1993). This work resulted from 
a major empirical analysis of training and performance outcome measures conducted by the United 
States Anny (“Project A,” Campbell, 1990) and has become somewhat of a gold standard in research 
and development projects in industrial and organizational psychology. The taxonomy posits eight 
broad, higher-order performance factors representing the range of outcomes that organizations value 
in their employees. The eight factors are intended to be distinct, and together, comprehensive. 
Although the system was developed and validated in an enlisted military context, it has been 
extended to apply to organizations more generally. The factors are sufficiently general that it seems 
reasonable to use the scheme as a way of organizing the factors important to graduate school 
success. Table 3 lists the factors, along with definitions and examples (Campbell et ah, 1993). 

Table 4 presents the categorization of the 21 attributes (from Table 2), along with seven 
competencies from Table 1 (Enright & Gitomer, 1989), into the eight higher-order performance 
factors from Table 3 (Campbell et ah, 1993). As can be seen, the categorization appears fairly 
natural, that is, the Campbell scheme is a good basis on which to sort attributes. Every factor has 
a highly endorsed attribute (a top 7) associated with it, except for Factor V. Factor II, Non-Job- 
Specific Task Proficiency, has the most attributes. Factor VIII, Management/Administration may 
simply be a factor more important in organizations than in graduate schools. Note also that 
affective characteristics are sprinkled over most of the perfonnance factors. 

Measures of Factors Important for Graduate School Success 

The discussion thus far has concerned what might be called the latent factors 
representing graduate school performance. A separate issue is how those abstract latent factors 
are actually measured. Table 5 lists many of the measures that have been used in previous 
validity research on higher education outcomes. What is striking in comparing Tables 4 and 5 is 
a mismatch between what faculty say are important outcomes, or graduate school success 
factors, and the convenience measures used in studies. Many of the convenience measures seem 
to be rather complex composites of the success factors. Grade point average might reflect many 
of the success factors—perhaps six of Campbell’s eight factors. Comprehensive exams are 
probably fairly clean measures of Job-specific task proficiency (or more appropriately, Domain- 
specific proficiency), but ratings, commendations, research productivity, career indicators, 
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attrition, and time to degree are undoubtedly complex composites. It may be that unpacking these 
variables into constituents like the eight factors, or into the even more specific 36 attributes, or 
something in between, might prove to be a useful approach for several reasons. It might facilitate 
providing graduate faculty the information they say they want about students, suggest more 
targeted interventions to increase specific graduate school outcomes, and enable more precise 
and informative outcome infonnation about the effects of those interventions. 


Table 3 


Campbell et ah, Higher Order Performance Components 


Factor 

Definition 

Examples (graduate student) 

Job-specific task proficiency 

Core tasks central to one’s job 

Knowledge of personality 
psychology methods and 
theories 

Non-job-specific task 
proficiency 

Required tasks that cut across 
manyjobs 

Teaching, literature reviewing, 
computer analyses 

Written and oral 
communication task 
proficiency 

Fonnal oral and written 
presentations, independent of 
content 

Writing research reports, 
presenting a paper at a 
conference 

Demonstration of effort 

Spending extra effort when 
asked, working in adverse 
conditions 

Working all weekend for a 
project due Monday, take on 
projects with tight deadlines 

Maintenance of personal 
discipline 

The degree to which negative 
behaviors are avoided 

Avoiding alcohol and drug 
abuse, ethical infractions, 
excessive absenteeism 

Facilitation of peer and team 
performance 

The degree to which one 
helps peers 

Help others with assignments, 
model goal directedness, 
involve others in group 

Supervision/leadership 

Supervisory tasks involving 
face to face interaction and 
influence 

Head a project, model first-year 
graduate students, set goals, 
reward and punish 

Management/ administration 

Supervision without face to 
face interaction 

Set goals for research group, 
establish timelines, monitor 
progress, recruit resources 
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Table 4 

Mean Importance Judgments of Outcome Variables Organized by Campbell’s Higher-Order 


Performance 

_ Outcome variables _ Mean rating 

I. Job-specific task proficiency 

1. Research/scholarly experience (amount and quality) 1.52 

5. Mastery of discipline 1.00 

11. Ability to read and analyze research in the field 0.43 

13. Breadth of perspective 0.29 


C. Explanation (Enright & Gitomer) 
G. Synthesis (Enright & Gitomer) 

II. Non-job-specific task proficiency 


7. Ability to teach (cross-listed in VII and VIII) 0.62 

8. Independence (cross-listed in IV) 0.43 

9. Creativity; ability to think “out of the box” 0.43 

14. Open mindedness 0.29 

15. Opportunism/resourcefulness 0.29 

16. Critical thinking ability, logic, problem solving ability 0.14 

18. Computer literacy/understand and manage technology 0.14 

19. Skill in investigation 0.14 

20. Ability to find the facts: information gathering 0.14 


B. Creativity (Enright & Gitomer) 

E. Planning (Enright & Gitomer) _ 

III. Written and oral communication task proficiency 


4. Communication (professional) 1.14 

6. English-as-a-second-language ability 0.71 

A. Communication (Enright & Gitomer) _ 

IV. Demonstration of effort 

2. Persistence/tenacity 1.48 

8. Independence 0.43 

10. Enthusiasm 0.43 

17. Drive/commitment/motivation/zeal 0.14 

D. Motivation (Enright & Gitomer) 
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Outcome variables 

Mean rating 

V. Maintenance of personal discipline 


12. Values/character: integrity, faimess/openness/honesty, 
trustworthiness/consistency—personal and 
professional 

0.29 

21. Professional posture 

0.14 

VI. Facilitation of peer and team performance 


3. Collegiality/making professional 
connections/networking 
(cross-listed in VII) 

1.38 

F. Professionalism (Enright & Gitomer) 


VII. Supervision/leadership 


3. Collegiality/making professional 
connections/networking 
(cross-listed in VI.) 

1.38 

7. Ability to teach (cross-listed in II and VIII) 

0.62 

VIII. Management/administration 


7. Ability to teach (cross-listed in II and VII) 

0.62 


Note. Based on Campbell et al. (1993). Ratings, where given, are repeated from Table 2 (see note 
for Table 2). 


Table 5 


Indicators of Graduate School Success That Have Been Used in Research Studies 


Indicator 


Citation 


Overall 

Kuncel, Hezlett, & Ones, 2001 

Grade-point average 

First year 

Kuncel et al., 2001 


Specific courses 

Goldberg & Alliger, 1992 

Comprehensive exams 


Kiman & Geisinger, 1981 


Student abilities 


Faculty ratings 

Dissertation quality 

Research 

Internship performance 

Kuncel et al., 2001; 

Sternberg & Williams, 1997 
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Indicator 


Citation 

Peer ratings 


Hirschberg & Itkin, 1978 

Commendations 


Nelson, 2000 

Research productivity 

Number of publications 

Citations 

Kuncel et al., 2001 

Career indicators 

Job offer 

Income & growth 

Cable & Murray, 1999; 

Clark & Centra, 1982; 

Employment setting 

Mobility 

Job satisfaction 

Crooks, Campbell, & Rock, 1979; 
Jamison, 1996 


Attrition 


Kuncel et al., 2001 

Time to degree 


Kuncel et al., 2001 


Predictors 

Walpole et al. (2001) asked faculty to identify variables that ought to be considered in 
admissions, as predictors. Table 6 presents the results. Fit is the most important (2.24), but a 
number of noncognitive variables rated quite highly, including values and character (1.14), 
maturity and responsibility (1.00), and interpersonal skills (.86). All these were mentioned more 
often than research experience (.86) and problem solving ability (.43). As Walpole et al. (2001) 
put it, a number of these attributes “are not well represented in current admissions folders.” 
According to the interviews, faculty attempt to glean some of this information from the personal 
statements applicants write, and from letters of recommendation. Table 7 lists the 30 attributes 
again, indicating which of these two sources faculty claimed to look to for this information. 

This provides a starting point for thinking about the kinds of noncognitive variables 
faculty members believe are important for admissions. For this review, there are three categories 
of predictors: personality, attitude, and quasi-cognitive. There are also two background 
categories: environmental factors and group factors. For each predictor category, there is a 
review of factors for that group, one by one, discussing their definition, measurement, 
consequents (variables they predict), and antecedents (variables by which they are predicted). 
Also reviewed are several kinds of measurement for each factor. A practical discussion of the 
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prospects for use of these measures in admissions concludes. These are described in the template 
used for reviewing factors shown in Figure 2. 

Table 6 


Mean Importance Judgments of Admission Variables 


Admission variables 

Mean rating 

Knowledge about program applying for, evidence of fit between 
student and program goals 

2.24 

Values/character: integrity, fairness/openness/honesty, 
trustworthiness/consistency—personal and professional 

1.14 

Maturity/responsibility/work habits/willingness to pay dues 

1.00 

Writing ability, intelligent writing 

0.95 

Research/scholarly experience (amount and quality) 

0.86 

Interpersonal skills (group/individual, work/social) 

0.86 

Commitment to field 

0.71 

Initiative/willingness to take a challenge 

0.67 

Breadth of perspective 

0.43 

Critical thinking ability, logic, problem solving ability 

0.43 

Drive/c ommitment/motivation/zeal 

0.43 

Ability to do advanced graduate work: masters? doctoral? 

0.43 

Leadership 

0.43 

Persistence/tenacity 

0.29 

Independence 

0.29 

Open mindedness 

0.29 

Communication (professional) 

0.14 

ESL ability 

0.14 

Creativity: ability to think “out of the box” 

0.14 

Computer literacy/understand and manage technology 

0.14 

Skill in investigation 

0.14 

Ability to find the facts: information gathering 

0.14 

Professional posture 

0.14 

Suited to environment in program applied for 

0.14 

Motivation to do research/scholarship 

0.14 
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_ Admission variables _ Mean rating 

Flexibility 0.14 

Positive attitude 0.14 

Professional contribution (e.g., teacher involvement with school) 0.14 

Ability to draw sound conclusions 0.14 

Ability to plan ahead _ 0.14 


Note. Rating is the mean mention level of the variable across the 21 participants, where 0 = not 
mentioned, 1 = conditional mention, 2 = moderate mention, and 3 = enthusiastic mention (from 
Walpole et ah, 2001, Table 1). Higher ratings indicate greater endorsement. 


Table 7 

Methods for Obtaining Information on Admission Variables 

_ Admission variables _ 

Knowledge about program applying for; evidence of fit between 
student and program goals 

Values/character: integrity, fairness/openness/honesty, 
trustworthiness/ consistency—personal and professional 

Maturity/responsibility/work habits/willingness to pay dues 

Writing ability, intelligent writing 

Research/scholarly experience (amount and quality) 

Interpersonal skills (group/individual, work/social) 

Commitment to field 

Initiative/willingness to take a challenge 

Breadth of perspective 

Critical thinking ability, logic, problem solving ability 
Drive/commitment/motivation/zeal (determination) 

Ability to do advanced graduate work: masters? doctoral? 

Leadership 

Persistence/tenacity 

Independence 

Open mindedness 

Communication (professional) 

ESL ability 


PS 


LR 
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Admission variables 


PS_ LR 

Creativity: ability to think “out of the box” 

Computer literacy/understand and manage technology 
Skill in investigation 

Ability to find the facts: information gathering 

Professional posture • 

Suited to environment in program applied for 

Motivation to do research/scholarship 

Flexibility 

Positive attitude 

Professional contribution (e.g., teacher involvement with school) 

Ability to draw sound conclusions (logic) • 

Ability to plan ahead _ 

Note. PS = personal statement, LR = letter of recommendation (from Walpole et ah, 2001, 
Appendices A and B). 


• Definition 

• Measurement 

Self-report—the typical survey or personality inventory 

Performance tests—a highly desirable, but rare find in the affective literature 

Ratings—by peers, faculty advisor, others 

Known groups—studying a construct by studying those who possess it 

• Factor as predictor 

• Predictors of factor 

Environmental factors 
Group factors 

• Prospects for applied use in admissions decisions 

Figure 2. Template for predictor variables. 
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Personality Factors 

In discussions of the role of noncognitive factors in graduate school, and their potential 
role in admissions, the factors that people usually have in mind are personality factors. 

Personality factors are those broad, stable personality traits (e.g., extraversion) assumed to 
operate consistently across a wide range of contexts. A theme in this review is that there are 
many other kinds of noncognitive factors. Still, because of their predominance in these kinds of 
discussions, it is appropriate to start with personality factors. 

Many personality schemes and taxonomies have been proposed in psychology, such as 
Cattell, Eber, and Tatsuoka’s (1970) 16-factor model (16 PF) and the Myers-Briggs typology, 
which is actually a four-factor model (Briggs & Myers, 1975). The plethora of schemes has 
made it difficult to appreciate the cumulativeness of personality research on a topic like 
predictors of graduate school success. Different investigators measure different factors with 
different instruments, making it difficult to perceive consistency in findings. Fortunately, a kind 
of consensus has emerged over the last decade that five basic factors underlie traditional 
personality assessment. These are extroversion, neuroticism (emotional stability), agreeableness, 
conscientiousness, and openness (Goldberg, 1992). The adoption of this scheme and the 
translation of older findings into a five-factor framework may lead to significant progress in our 
understanding of the relationship between personality and graduate school perfonnance. 

It is important to understand why, in a field as contentious as personality psychology, a 
consensus has emerged at all. We believe the reason is really twofold. First, the five-factor 
structure has been found across cultures and language groups (McCrae, Costa, Del Pilar, Rolland, 
& Parker, 1998). The five-factor structure is not unique to the United States, nor is it unique to the 
English language. Second, there is a compelling idea underlying the empirical discovery of five 
replicable factors. The idea is what Goldberg refers to as “the lexical hypothesis,” which is that 
language evolves to produce words that encompass the most important or useful dimensions by 
which we characterize people. In the English language there are thousands of such descriptive 
personality adjectives. A reduction of those thousands of words, which can and has been 
accomplished with factor analyses of person ratings on all the personality-descriptive adjectives in 
a language, should yield the most basic, fundamental dimensions of personality. It is from studies 
done exactly this way that the five-factor model has emerged. 
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An important, but under-appreciated implication of this model is that other general 
personality dimensions, such as self-monitoring, alexithymia, machiavelianism, and locus of 
control are really for the most part reducible to, or represent weighted combinations of these five 
factors. In fact, considerable evidence has been amassed to demonstrate this (Goldberg, in press). 
In general, if the five-factor model is correct, any broad personality factor, identified through a 
self or peer assessment of trait adjectives, phrases, sentences, or paragraphs, or other depictions 
(e.g., video vignettes), should be accounted for by the Big 5. 

Does that mean there is no role for personality constructs other than the Big 5? No, there 
are at least two kinds of variables that while fitting within the Big 5 scheme are not the Big 5 
variables, and yet may be useful for both research and applied purposes. One kind is compound 
variables, that is, variables that represent particular combinations of the Big 5. Examples are those 
given in the previous paragraph. Snyder’s (1987) Self-monitoring may be a combination of 
extroversion (+), agreeableness (+), and conscientiousness (-), 1 for example. Authoritarianism 
(Adorno, Frenkel-Brunswik, Levinson, & Sanford, 1950) might be a combination of 
conscientiousness (+), agreeableness (-), and openness (-) (these two examples from Funder, 
2001). Psychoticism (Eysenck & Eysenck, 1968) might be a combination of conscientiousness (-) 
and agreeableness (-). Similar compound variables, or “blends,” such as Poise (extroversion [+] 
and neuroticism [-]), Provocativeness (extroversion [+] and agreeableness [-]), and Perfectionism 
(conscientiousness [+] and neuroticism [+]) are also featured in the Abridged Big-5 Circumplex 
Model (AB5C) (Hofstee, de Raad, & Goldberg, 1992). For particular purposes, and perhaps for 
understanding, the compound variables may in some circumstances be quite useful, even more so 
than the five factors themselves, although it is not at all clear that this represents a lack of 
comprehensiveness to the Big 5 scheme, as some have argued (Funder, 2001). 

Another kind of variable that may usefully supplement the five factors are the sub¬ 
factors, or facets of the Big 5, such as excitement-seeking, friendliness, and assertiveness, which 
are facets of extroversion. These are not compounds or blends as much as they are shades of 
meaning (or technically, correlated dimensions within each) of the five factors. Again, in some 
situations, these facets may be more important (i.e., valid) than the five factors themselves 
(Paunonen & Ashton, 2001). It is important to note that unlike the Big 5, there is no 
standardization on the underlying facets: There are potentially hundreds if not thousands of them. 
The ones listed in Table 8 (which also presents items for each facet, one positive and one 
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negative) are listed because they happen to come from a popular Big 5 inventory, the NEO 
(Costa & McCrae, 1992). They represent a reasonable sense of the kinds of facets that can be 
found within the Big 5. 


Table 8 


Big 5 Factors, Facets, and Items 


Big 5 Factor 

Sample item from IPIP 

facet 

Positive item 

Negative item 

Extraversion 

Friendliness 

Make friends easily 

Am hard to get to know 

Gregariousness 

Love large parties 

Prefer to be alone 

Assertiveness 

Take charge 

Wait for others to lead the way 

Activity level 

Am always busy 

Like to take it easy 

Excitement seeking 

Love excitement 

Would never go hang gliding 

Cheerfulness 

Radiate joy 

Am not easily amused 

Neuroticism 

Anxiety 

Fear for the worst 

Not easily bothered by things 

Anger 

Get angry easily 

Rarely get irritated 

Depression 

Often feel blue 

Rarely feel blue 

Self-consciousness 

Am easily intimidated 

Am not embarrassed easily 

Immoderation 

Often eat too much 

Rarely overindulge 

Vulnerability 

Panic easily 

Remain calm under pressure 

Conscientiousness 

Self-efficacy 

Complete tasks smoothly 

Misjudge situations 

Orderliness 

Like order 

Forget to put things back 

Dutifulness 

Try to follow the rules 

Break rules 

Achievement striving 

Go straight for the goal 

Not highly motivated 
to succeed 

Self-discipline 

Get chores done right away 

Waste my time 

Cautiousness 

Avoid mistakes 

Jump into things without 
thinking 
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Big 5 Factor 

Sample item from IPIP 

facet 

Positive item 

Negative item 

Agreeableness 

Trust 

Trust others 

Distrust people 

Facet 

Positive item 

Negative item 

Morality 

Wouldn’t cheat on taxes 

Use flattery to get ahead 

Altruism 

Make people feel welcome 

Look down on others 

Cooperation 

Am easy to satisfy 

Have a sharp tongue 

Modesty 

Dislike being center of attention 

Believe I am better 
than others 

Sympathy 

Sympathize with homeless 

Not interested in others’ 
problems 

Openness to experience 

Imagination 

Have a vivid imagination 

Seldom daydream 

Artistic interests 

Believe in importance of art 

Do not like art 

Emotionality 

Experience emotions intensely 

Seldom get emotional 

Adventurousness 

Prefer variety to routine 

Stick to things I know 

Intellect 

Like to solve complex problems 

Not interested in abstract 
ideas 

Liberalism 

Vote for liberal political 
candidates 

Believe in one true religion 


Note. From the International Personality Item Pool [IPIP] (n.d.). Items are administered with 
instructions to use a rating scale (1 = very inaccurate to 5 = very accurate) to “describe how 
accurately each statement describes you ... as you generally see yourself now, in relation to 
other people you know of the same sex as you are and roughly the same age.” Some items 
slightly abridged to fit in columns. 


Other general personality factors have been proposed. Among these are optimism 
(Seligman, 1991), integrity (Ones, Viswesvaran, & Schmidt, 1993), and typical intellectual 
engagement (Goff & Ackerman, 1992). Their independence from the Big 5 has not been 
convincingly demonstrated (Goldberg, 1999), but because they have a literature surrounding 
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them, and for convenience, they will be included in the Big 5 categories of extroversion, 
conscientiousness, and openness, respectively. 

There are also factors that legitimately appear to be outside the Big 5, primarily in the 
trait terms used to rate them, which might not appropriately be called personality descriptors. 
These include social-attitude factors like religiosity, conservatism, authoritarianism, and 
spiritualism (Saucier, 2000), and non-personality person-descriptive factors such as youthfulness, 
fashionableness, sensuality, frugality, humor, and luck (Saucier & Goldberg, 1998). Some of 
these appear to have some empirical relationship with the Big 5. Social-attitude factors are 
discussed further in the Attitudinal Factors section. 

Definition of the Big 5 Factors. Five basic factors underlie traditional personality 
assessment: extroversion, neuroticism (emotional stability), agreeableness, conscientiousness, 
and openness (Goldberg, 1992). 

Extraversion. Extraversion is the “people person” factor representing outgoingness and 
sociability, but also factors such as demonstrating independence, decisiveness, and orientation to 
negotiating. Facets include friendliness, gregariousness, assertiveness, activity level, excitement 
seeking, and cheerfulness. Hogan (1986) and Hough (1997) suggest that this factor can be 
divided into sociability or affdiativetiveness on the one hand (e.g., friendliness, gregariousness, 
cheerfulness), and ambition or potency on the other (e.g., assertiveness, activity level, excitement 
seeking). The factor of optimism is likely related to extraversion—it is a scale in the IPIP 
(Goldberg, 2001), and is related to the well-being scale in the California Psychological Inventory 
(CPI) (Gough, 1975). However, the construct has received lots of special attention due to a 
popular book by Seligman (1991). 

Neuroticism (emotional stability). Neuroticism is the factor representing emotional 
stability vs. instability. An emotionally stable student would be one to respond well to stress and 
tight time deadlines, be adaptable, and take all information into account when making decisions, 
particularly under stress or deadlines (Goldberg, 2001). Facets include anxiety, anger, 
depression, self-consciousness, immoderation, and vulnerability. It may be useful to note that in 
Eysenck’s personality theory formulated in the 1950s and 1960s, extraversion and neuroticism 
were the two primary factors. 

Conscientiousness. Conscientiousness is a measure of the degree to which one is 
organized, follows schedules, performs administrative tasks accurately and thoroughly, follows 
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regulations and procedures, is disciplined, demonstrates integrity and adherence to ethical 
standards, is persistent and motivated, and exerts effort (Goldberg, 2001). Hough (1992, 1997) 
has suggested that his factor may be subdivided into two factors: dependability, that is, being 
careful, thorough, responsible, planned and organized, and achievement striving, that is, being 
perseverant and hard working. 

Integrity. Integrity may be understood as a facet of conscientiousness, or as a compound 
variable consisting of conscientiousness, agreeableness, and adjustment(Ones & Viswesvaran, 
1998). However, because of its important practical role in employment testing, it is often 
measured separately from conscientiousness, with its own assessments. Consequently, it has 
been treated separately from conscientiousness in metaanalytic studies of the validity of 
employment predictors (e.g., Ones, Viswesvaran, & Schmidt, 1993; Schmidt & Hunter, 1998). 
Integrity testing measures one’s propensity towards counterproductive work behaviors, such as 
theft, drug and alcohol use, undependability, and causing property damage, through accidents or 
otherwise (Ones & Viswesvaran, 1998; Sackett & Wanek, 1996). Three of the most widely used 
integrity scales are the PDI Employment Inventory (Paajanen, 1985), the London House 
Employment Productivity Index (Rafilson, 1988); and the Reliability Scale of the Hogan 
Personnel Selection Series (Hogan & Hogan, 1989) (scales for the Big 5 are covered below). 

Agreeableness. This factor has also been called likeability, friendliness, social 
conformity, and love. It is defined as being cooperative and considerate, treating others fairly, 
and with kindness, maintaining harmony, being tolerant, courteous, flexible, good-natured, and 
recognizing and rewarding others’ performance. Facets include trust, morality, altruism, 
cooperation, modesty, and sympathy. 

Openness-to-experience. This factor is also referred to as intellect and culture. It is 
considered the loosest or most varied factor, representing the degree to which one is open- 
minded, curious, creative, knowledgeable, analytic, and able to communicate effectively 
(Goldberg, 2001). Facets include imagination, artistic interests, emotionality, adventurousness, 
intellect, and liberalism. Hogan and Hogan (1995) differentiate the construct into intellectance 
(creativity) and school success (achievement orientation). Typical intellectual engagement (Goff 
& Ackerman, 1992), and need for cognition (Cacioppo, Petty, Feinstein, & Jarvis, 1996) are 
likely related factors. 
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Measurement. Several ways exist to measure the Big 5 Factors: self-reports, ratings, 
performance tests, and known groups. 

Self-reports. Many multiple-scale personality inventories have been developed over the 
years. Although scales vary widely, all may measure at least some of the Big 5 Factors. Among 
the most widely used are the California Psychological Inventory (CPI) (Gough, 1975), Comrey 
Personality Scales (CPS) (Comrey, 1970), Edwards Personal Preference Schedule (EPPS) 
(Edwards, 1959), Eysenck Personality Questionnaire (EPQ) (Eysenck & Eysenck, 1975), Gordon 
Personal Profile Inventory (GPPI) (Gordon, 1978), Guilford-Zimmerman Temperment Survey 
(GZTS) (Guilford, Zimmerman, & Guilford, 1976), Hogan Personality Inventory (HPI) (Hogan 
& Hogan, 1995), Jackson Personality Inventory (JPI) (Jackson, 1976), Minnesota Multiphasic 
Personality Inventory (MMPI) (Dahlstrom, Welsh, & Dahlstrom, 1972), Multidimensional 
Personality Questionnaire (MPQ) (Tellegen, 1982), Myers-Briggs Type Indicator (MBTI) 
(Briggs, Myers, & McCaulley, 1985), Omnibus Personality Inventory (OPI) (Heist & Yonge, 
1968), Personality Research Fonn (PRF) (Jackson, 1976), and the Sixteen Personality Factor 
Questionnaire (16 PF) (Cattell et ah, 1970). 

Perhaps the most widely used Big 5 instrument per se is the revised NEO and its variants, 
(Costa & McCrae, 1992), such as the NEO-IP. Goldberg (1999) has put into the public domain 
an extensive set of personality items measuring the Big 5 factors and their facets (IPIP, n.d.). He 
calls the project the International Pool of Personality Items (IPIP). This is an amazingly useful 
resource, likely to have a significant impact on research progress, as researchers are now free to 
assemble five-factor inventories for just about any purpose, free of charge. 

Ratings. Inventories such as the NEO and the IPIP are typically administered as self- 
ratings, but peer (and supervisor and teacher, etc.) ratings are equally possible, and in fact have 
been employed since the earliest formulations of the five-factor model (Norman, 1963; Tupes & 
Christal, 1958). The Big 5 factor structure is similar regardless of whether it is based on peer or 
self ratings, but there can be discrepancies between peer and self ratings (Lanthier, 2000). 
Discrepancies, although minimized with familiarity with the subject (Paunonen, 1989), may in 
themselves be informative. Mount, Barrick, and Strauss (1994) showed that others’ ratings added 
to self-ratings in incrementing the validity of personality constructs for job perfonnance. 

Performance tests. There have been attempts to develop performance tests (also called 
objective-analytic, or ability tests) for measuring general personality factors over the years (e.g., 
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Cattell & Schuerger, 1978), but they have not met with tremendous success. A number of studies 
(e.g., Kinney, 1999) have tried variants on reaction time methods, such as latency to endorse a 
trait adjective as self-descriptive, on the assumption that short latencies indicate more intensity 
(positive or negative endorsement) with respect to that trait term. Another method has been word 
recognition time, under the assumption that more personally salient terms are recognized more 
quickly. However, reaction times reflect other processes, such as mental speed, and one’s 
familiarity with the term, which confounds the interpretation. 

A potentially more promising approach to a performance personality test is the 
conditional reasoning paradigm developed by James (1998). This involves presenting a reading 
passage with a multiple-choice problem that looks like a typical reasoning problem one might 
find on something like the GRE General Test (reading comprehension). However, the problem 
presents two equally correct answers (of five alternatives). The idea is that one’s choice from 
among the two correct answers reflects justification mechanisms one habitually employs, which 
reflect something about one’s personality. For example, James suggests that achievement 
motivation can be reflected by two quite different dispositional tendencies—the motive to 
achieve, and the fear of failure—and that individuals habitually favor one over the other. A 
conditional reasoning problem can be written to detennine which tendency a particular 
individual favors. 

As an example James (1998) provides a passage that describes the relationship between 
stress, heart disease, and Type-A personality. It then asks the question, “Which of the following 
would most weaken the prediction that striving for success increases the likelihood of a heart 
attack?” Three of the alternatives are wrong on logical grounds, but two, according to James, are 
equally valid. One is that “A number of nonambitious people have heart attacks.” The other is 
that “Recent research has shown that it is aggressiveness and impatience, rather than 
achievement motivation and job involvement, that are the primary causes of high stress and heart 
attacks.” According to James, those motivated by fear of failure choose the former, because they 
agree with the passage’s conclusion (that success striving increases heart attack risk), and they 
opt for the wounding response alternative (something that does not seriously invalidate the 
conclusion). On the other hand, achievement motivated individuals choose the latter, because it 
is based on a more positive connotation of achievement striving. 
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James (1998) presents an additional example to measure aggressiveness, and suggests 
that his general method could be used to develop items for other personality constructs. He also 
presents some empirical evidence supporting the validity of his approach. However, the approach 
and the items yielded by it are obviously in their infancy, and only further research will 
demonstrate whether this is a viable method for measuring personality. 

Known groups. The known groups method involves identifying a group acknowledged to 
be high (or low) on a particular construct (e.g., creativity), and then measuring various attributes 
of the known group as a way to study the construct (e.g., creative people are verbally fluent). 

This method has only indirectly been employed in studying personality, through empirical 
response keying accompanied by psychiatric diagnosis, as in the MMPI (Dahlstrom et ah, 1972). 

However, a perhaps somewhat related method is the open-ended personal narrative 
(McAdams, 1999). This method involves participants describing significant events in their lives, 
such as life-story high points, low points, turning points, and earliest memories (McAdams, 
Reynolds, Lewis, Patman, & Bowen, 2001), and coders interpreting those narratives for 
personality imagery. The coding is critical. McAdams et al. found that the specific coding of 
narratives for redemption and contamination imagery was more predictive of well-being than 
was an overall affective assessment of the narrative. Like James’ (1998) conditional reasoning, 
McAdams’ personal narrative approach is a relatively novel methodology, and it is not clear how 
or whether it might be translatable to other personality constructs. It is worth noting that 
candidates routinely submit personal narratives of sorts to graduate admissions committees in the 
form of personal statements. Perhaps developments in methodology for scoring personal 
narratives could inform our analysis of personal statements. 

Big 5 Factors as predictors. There are few large-scales studies on the relationship 
between personality and success in higher education. However there have been several meta¬ 
analyses of the relationship between personality and job training and job performance. 
Conscientiousness is probably the most consistently strong predictor of job perfonnance, 
regardless of the job, followed by emotional stability, then extraversion (Barrick & Mount, 1991; 
Barrick, Mount, & Judge, 2001; Hurtz & Donovan, 2000; Salgado, 1998; Tett, Jackson, & 
Rothstein, 1991), and sometimes agreeableness, particularly for jobs involving teamwork 
(Mount, Barrick, & Stewart, 1998). Because personality is independent of cognitive ability (with 
the exception of a modest correlation between cognitive ability and openness), personality 
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validities for job performance, particularly conscientiousness and integrity, are incremental 
(Schmidt & Hunter, 1998). The magnitude of incremental validity is significant—approximately 
in the .10s and .20s. Several have suggested that differentiating outcome criteria yields even 
stronger relationships (Hurtz & Donovan, 2000). For example, Hogan and Holland (2002) 
suggest that dividing job criteria into the categories of getting along (e.g., helping others) and 
getting ahead (e.g., coming to work early and leaving late), result in higher validity coefficients 
than are obtained when lumping all criteria together, and the magnitude runs from the .30s to 
.40s (for all factors). These are clearly substantial validities considering that they are 
uncorrelated with cognitive ability. 

There have been several small studies relating personality variables to educational 
variables. They mostly confirm the general findings from the industrial-organizational literature, 
that conscientiousness is important (e.g., Busato, Prins, Elshout, & Hamaker, 2000; Hirschberg 
& Itkin, 1978; Van-Heyningen, 1997), and that the other personality variables can be important 
depending on the criterion (e.g., Chemers, Hu, & Garcia, 2001; Musgrave-Marquart, Bromley, & 
Dailey, 1997; Brown & Marshall, 2001). However, there have been no meta-analyses of this 
literature (and at this point there does not seem to be enough data on which to conduct a meta¬ 
analysis). 

It may seem puzzling why so few validity studies of personality on higher education 
outcomes have been conducted, given the demonstration of the importance of personality in 
work settings. Goldberg (2001) and Reeve and Hakel (2001) provide specific suggestions for 
how personality variables (e.g., factors, facets, and items) reflect specific valued graduate school 
outcomes. Both suggest that conscientiousness and emotional stability ought to predict graduate 
school outcomes across the board, with extraversion, and perhaps agreeableness predicting 
success in particular academic disciplines (they do not say so, but one can speculate that 
teaching, nursing, and clinical disciplines might show such relationships). Of course, openness is 
likely to be related to academic outcomes, too, but it is the one personality factor consistently 
correlated with general cognitive ability—it typically fails to show incremental validity over 
general cognitive ability in studies where both factors are included. 

Goldberg (2001) suggests that the reason for the shortage of validity studies is that 
university selection is higher stakes than employment selection, which would lead to higher 
scrutiny, and that educators fear that “self-reports are far too amenable to impression 
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management to be used in high-stakes contexts” (p. 11). He counters by pointing out literature 
suggesting that the importance of impression management is overblown (Ellingson, Smith, & 
Sackett, 2001; Smith, Hanges, & Dickson, 2001), a point we return to in the Prospects for 
Applied Use in Admissions Decisions section. 

Predictors of the Big 5 Factors. Among the predictors of the Big 5 are environmental 
factors and group factors. 

Environmental factors. For the purpose of this review, we treat personality as an 
independent variable, unaffected by environmental factors (see Figure 1). This is in keeping with 
the current consensus of opinion, which suggests that the broad personality factors discussed in 
this section are relatively stable over time (e.g., Heath, Neale, Kessler, Eaves, & Kendler, 1992). 
This is not to say that specific environmental factors (e.g., repeated and prolonged exposure to 
stressful events) cannot affect personality (e.g., neuroticism)—this is a complex topic about 
which surprisingly little is yet known (Brody & Crowley, 1995). It is just that intervention 
studies on personality factors have not been conducted, and so we know almost nothing about 
changes in personality. In this review, we differentiate the broad and stable factors covered in 
this section from more specific factors, which are likely more malleable. These more specific 
factors are discussed in the Motivation Factors section later in this paper. 

Group factors. The same basic five-factor structure is found across different language 
groups. The following are some examples: Greek (Tsaousis, 1999), Spanish (Benet-Martinez & 
John, 1998), German, and Dutch (De Raad & Hendriks, 1997); ethnic groups (e.g., White, 

Black, Indian, and mixed-race South African college students) (Heuchert, Parker, Stumpf, & 
Myburgh, 2000); and groups within the United States (e.g., Black and White students) (Collins 
& Gleaves, 1998). This discredits the idea that the big-five taxonomy is a perspective on 
personality unique to western culture, to the United States, or to the majority racial/ethnic group 
within the United States. A Big 5 measure is available in numerous languages, including, in 
addition to those mentioned above, Brazilian, Chinese, Croatian, Hebrew, Hungarian, Italian, 
Japanese, Polish, Slovak, and Swedish (Hendriks, Hofstee, & de Raad, 1999). 

There seems to be little if any difference in means between gender or racial groups on 
personality factors or facets (Collins & Gleaves, 1998; Goldberg, Sweeney, Merenda, & Hughes, 
1998). Heuchert et al. (2000) found a small mean difference between White, Indian, and Black 
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students from South Africa in openness, particularly the openness to feelings facet, which they 
attributed to cultural differences. 

Prospects for applied use in admissions decisions. In this section we consider 
practical issues associated with using personality measures in admissions decisions. There are 
numerous ways personality measures could be used: We can classify them into guidance uses, 
and selection uses. 

A major issue in using personality in selection is that personality is primarily measured 
through self-reports. Consequently, there is a question about the degree to which personality tests 
are susceptible to faking. Reeve and Hakel (2001) suggested three approaches to detecting and 
controlling faking. Detection and correction involves mixing lie scale (or social desirability 
scale) items in with other personality items. Such items are highly sensitive to instructions to 
fake good, and respondents who are shown to score highly on such items can have their 
responses to the other personality items statistically adjusted. The problem with this method is 
that even with adjustment, it is almost always advantageous to fake good (Ellingson, Sackett, & 
Hough, 1999), and more generally, with open disclosure, it would likely be possible for coaching 
schools to figure out a way to beat the scoring adjustment system. 

A second procedure is the warning method, in which the proctor warns examinees that 
their score patterns will be scrutinized and adjusted if found to be dishonest. Although this 
method has proven successful in research, it is unlikely that the warnings would have any effect 
in a large-scale, high-stakes setting, particularly for some individuals. Some test takers go to 
extraordinary lengths to cheat on the test, even risking arrest (e.g., Carnevale, 2002). 

A third method is the subtle-items method, involving the use of items that bear no 
obvious relationship to the factor being measured, such as the kind of empirically keyed items 
used on the MMPI (e.g., “Do you like red cars?”). The problems are that such items are typically 
less valid, and, perhaps more importantly, may be vulnerable to legal challenges questioning the 
connection between responses on such items and perfonnance in graduate school. An important, 
possible exception to these limits are the kinds of conditional reasoning items developed by 
James (1998). Although subtle, such items do connect to the factor being measured through a 
rational, if complex, chain of reasoning, which presumably would fare well in a court challenge. 
Whether such items are as valid as more typical personality items remains to be seen. And 


26 



whether the hundreds or even thousands of items necessary for a high-stakes test could be 
developed also remains to be seen. 

A fourth method is the multidimensional forced-choice method, involving the pitting of 
two or more presumably equally attractive, but quite distinct trait statements, to which the 
examinee must indicate a preference. This is an ipsative approach to personality measurement, 
and cannot be assumed to produce scores similar to the normative scores on which most of the 
known research in personality is based. Thus this must be considered only a potentially 
promising method. 

If somehow the intentional response distortion effect could be controlled, there would 
remain the problem of generating items for new test forms. There are many ways to express 
personality items, limited only by an item writer’s creativity in devising situations and 
expressing those in an accessible vocabulary. However, it is not clear that the realm of items and 
situations would be large enough to disguise the underlying construct being asked for. There may 
be millions of ways to ask whether an applicant is dependable and hard working. Still, once an 
applicant knew that conscientiousness is what he or she was being tested for, might not any 
question that tapped it be obvious? 

Using personality for guidance—to help a candidate make a good decision about whether 
and what kind of graduate school program to attend—avoids all the problems associated with 
using personality for selection. Coachability, fakability, and item security are simply not issues, 
because the candidate has no reason to fake responses. 

Another way to possibly avoid the problems of selection is through the use of others’ 
rating of personality. Others could be advisors, professors, and other members of the college 
community who typically write letters of recommendation for students now. Although they too 
could be taught to game the system and fake good for their applicant ratee, it would seem that 
that problem is no greater or less than the current problem of distortion in letters of 
recommendation. One problem that might be greater with such a scheme than it is with current 
letters of recommendation is one of increased scrutiny, and perhaps legal exposure. Current 
letters typically use circumspect language, which a personality form fdled out by a faculty 
advisor would explicitly avoid. 

In any event, there is a lure in using personality as part of the admissions process. As 
Sackett, Schmitt, Ellingson, and Kabin (2001) and Goldberg (2001) point out, the laudable goals 
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of both increasing ethnic diversity and assessing the full range of relevant attributes are pursued 
by adding something like a Big 5 personality assessment to the mix of selection materials. Until 
a way around the major problem of intentional response distortion can be discovered, perhaps 
through ability-test or perfonnance-test methods such as those James (1998) has developed, it is 
unlikely that personality can be measured through self reports. However, other reports, provided 
by the same supervisors and mentors currently providing letters of recommendation, might 
present an attractive alternative for systematically introducing personality information into the 
admissions process. 

Quasi-Cognitive Factors 

Although this review concerns noncognitive factors, the scope is broadened somewhat to 
include four quasi-cognitive factors—creativity, emotional intelligence, cognitive styles, and 
metacognition. There are two reasons for this. First, these factors, at least the first two, are often 
brought up in discussions of important factors not measured by tests like the GRE General Test. 
Second, each may indeed include a significant affective or noncognitive component. 

Creativity. Creativity is a difficult factor to define and operationalize, in part because 
researchers disagree on what it is (e.g., Torrance, 1988). Some have focused on creative people 
and defined creativity as the abilities found in such people (e.g., Guilford, 1950). Others have 
focused on creative products, believing that studying them will produce insights into the nature 
of creativity (e.g., Bruner, 1962; Jackson & Messick, 1965). One definition of whether a 
product is creative is whether “appropriate observers independently agree it is creative” 
(Amabile, 1996, p. 33). 

Others see creativity as a conflux of different variables that must be present for a creative 
act to occur. These variables can include the correct environment, sufficient intelligence and 
motivation, and thinking and personality styles that are conducive to creative behavior (e.g., 
Amabile, 1996; Csikszentmihalyi, 1996; Sternberg & Lubart, 1996). 

Measurement of creativity. There are several ways to measure creativity: self-reports, 
biographical data, performance tests, ratings/product ratings, known groups, and observation. 

Self-reports. A way to find out if someone is creative is to have the person estimate his or 
her own creativity (e.g., Fumham, 1999b). Indeed, many major personality assessments (e.g., the 
California Psychological Inventory; Myers-Briggs Type Indicator) include creativity indexes. 
Other methods include providing brief characterizations of creative individuals that examinees 


28 



are then asked to identify as being similar or dissimilar to themselves (the Self-Descriptive 
Creativity Test, Smith, & Faeldt, 1997), and examining beliefs about and strategies used for 
creativity (The Creativity Styles Questionnaire, Kumar & Holman, 1989). 

Biographical data. Accomplishments such as awards (Kaufman, 2000-2001), length of 
career (Crozier, 1999), age at different accomplishments (Simonton, 1988), influence, 
productivity, and international recognition (Ludwig, 1995) have been examined in studies of 
eminent creative individuals. Biographical data has also been used to study everyday creativity 
for example, through lists of creative accomplishments (King, McKee-Walker, & Broyles, 1996), 
or questions about past creative performance (Hocevar, 1979; Taylor & Ellison, 1966). 

Performance tests. The popular Torrance (1974) Tests of Creative Thinking (TTCT) 
include both verbal items, for example, showing participants a picture or a product and asking a 
series of questions about it; and figural items, for example, presenting an idea (the caterpillar 
with too many legs) and then asking participants to draw a figure based on the idea. The test is 
scored using a rubric that rewards unusual responses. Many of Guilford’s (1967) divergent 
thinking measures are similar, allowing respondents to generate elaborations of a given shape, or 
hypotheses about different outcomes of an event (e.g., if people didn’t need sleep). Similar tests 
have been designed by others (Getzels & Jackson, 1962; Wallach & Kogan, 1965). Finke, Ward, 
and Smith (1992) developed a slightly different approach. They asked participants to create 
inventions or devices that are then judged by experts for creativity or other attributes. 

Ratings/product ratings. Studies have used creativity ratings of products by teachers 
(Besemer & O’Quin, 1986) or parents (Runco & Vega, 1990). Amabile (1996) developed a 
Consensual Assessment Technique (CAT), in which raters, solicited for their experience in a 
domain, provide independent judgments about the creativity of a product. Raters compare 
products against each other, rather than against an absolute ideal, and the products are judged in 
a random order (Amabile, 1985; Amabile, Hennessey, & Grossman, 1986; Conti, Coon, & 
Amabile, 1996). One concern of this method is that raters from different cultures can differ on 
their perceptions of creativity (Niu & Sternberg, 2001). 

Known groups. Another method of examining creativity is to identify a group of people 
who are known to be creative and study their common traits or features. Some researchers have 
examined a wide sample of kn own groups of eminent individuals from a variety of professions 
(e.g. Ludwig, 1995; see Simonton, 1994). Others have identified subgroups thought to be 
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particularly creative, such as actors (Hammond & Edelmann, 1991), architects (Hall & 
MacKinnon, 1969), cinematographers (Domino, 1974), advertising directors (Broyles, 1996), 
musicians (Wills, 1984), and creative writers (Kaufman, 2001a). 

Observation. Observation is rarely used in the measurement of creativity, in part because 
successful creative people will rarely allow themselves to be observed while working and in part 
because it is difficult to leam very much from watching someone during a creative activity. An 
exception was a study by Dunbar (1995), who spent a year observing scientists work in 
laboratories. Through interviews, observations, video and audiotaping, he found that creative 
scientists did not often experience sudden insight, but rather proceeded along a very careful and 
well-reasoned mode of study. 

Predictors of creativity. In his structure of cognitive abilities, Carroll (1993) assigns 
creativity to his second-stratum factor called broad retrieval abilities, which also encompasses 
fluency measures (e.g., write all the five-letter words that begin with “s” and end in “g” in two 
minutes). The focus here is on creativity’s li nk s to other noncognitive factors. 

Personality factors. Openness to experience correlates with creativity. This has been 
found with self-reports of creative acts (Griffin & McDermott, 1998), biographical data on 
creative accomplishments (King, McKee-Walker, & Broyles, 1996), creative professions 
(Domino, 1974), analyzing participants’ daydreams (Zhiyan & Singer, 1996), creativity ratings 
on stories (Wolfradt & Pretz, 2001), and psychometric tests (Fumham, 1999a; McCrae, 1987). 

Artistic creativity has been found to be negatively associated with conscientiousness 
when studying creative professions (Dudek, Bemeche, Berube, & Royer, 1991) and using 
biographical data (Walker, Koestner, & Hum, 1995); however, no such relationship exists for 
scientific creativity. If anything, the tendency is in the opposite direction (see Feist, 1999). There 
may be a possible interaction between openness to experience and conscientiousness; among 
people who are high in openness to experience, conscientiousness may reduce creativity as 
scored in a test of fluency (Ross, 1999). 

There may also be a mild link between creativity and extraversion. Martindale and Dailey 
(1996) had participants write a fantasy story, which was then rated for creativity, and also 
administered psychometric tests; they found a significant, positive relationship with extraversion. 
This relationship was also found in an analysis of biographical data, controlling for academic 
accomplishments (King et ah, 1996) and using psychometric tests (Richardson, 1985; Srinivasan, 
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1984) . Yet Roy (1996), in a study of fine artists (in comparison to other professions), found that 
the artists were more introverted than the controls; Mohan and Tiwana (1987) found similar 
results with a group of creative writers. Several other studies (most notably Matthews, 1986; 
McCrae, 1987) found no significant relationship at all. 

Motivation factors. Some attention has been given to the relationship between motivation 
and creativity, particularly the distinction between intrinsic and extrinsic motivation. A common 
claim is that intrinsic motivation (performing an activity for enjoyment) is more conducive to 
creative work than extrinsic motivation (perfonning an activity for a reward). This seems to be 
true whether motivation is natural (task involvement; Ruscio, Whitney, & Amabile, 1998) or 
induced experimentally, for example by reading about extrinsic reasons for writing (Amabile, 

1985) . Some have argued that the case against extrinsic motivation is overstated (Eisenberger & 
Cameron, 1996). Rewards can improve perfonnance if the criterion task requires divergent 
thinking (Eisenberger & Selbst, 1994), the instructions emphasize the need for creativity 
(Eisenberger, Armeli, & Pretz, 1998), or students are experienced with creative acts 
(Eisenberger, Haskins, & Gambleton, 1999). These studies measured creativity through product 
ratings. 

Environmental factors. Social support may be important to creative work (Bargar & 
Duncan, 1990). Having supportive teachers (Cole, Sugioka, &Yamagata-Lynch, 1999) or friends 
as collaborative partners (Miell & MacDonald, 2000) has been shown to lead to products judged 
more creative. 

Group factors. Women and men score similarly on creativity tests regardless of culture 
and background (see Barron & Harrington, 1981; Saeki, Fan, & Van Dusen, 2001; Wang, Zhang, 
Lin, & Xu, 1998). For Black and White students, there have been no significant differences in 
creative performance on the TTCT (Glover (1976a, 1976b), ability to be trained on creative 
abilities (Moreno & Hogan, 1976), and in biographical measures of aesthetic expression 
(Strieker, Rock, & Bennett, 2001). 

Results are less clear for other cultures. American college students scored higher on the 
TTCT than Japanese college students (Saeki et. ah, 2001). Malaysian students scored higher than 
American, Indian, and Hungarian students on one self-report measure of creativity, but American 
students scored higher than Malaysian students on a different self-report measure (Palaniappan, 
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1996). One problem with assessing creativity in other cultures is that the definition of creativity 
may be culture-specific (see Lubart, 1999). 

Criterion factors. While creativity (as measured by psychometric tests such as the TTCT) 
is significantly correlated with intelligence, particularly verbal intelligence (e.g., Barron & 
Harrington, 1981), the correlation is not especially strong. In addition, correlation with 
intelligence is only true up to a point (usually found to be approximately an IQ of 120; Getzels & 
Jackson, 1962). Similarly, creativity as measured by the TTCT accounted for some variance in 
academic performance, but less so than other cognitive variables (e.g., Niaz, Saud de Nunez, & 
Ruiz de Pineda, 2000). 

Creativity scores have been shown to predict persisting in difficult situations (Falat, 
2000), more effective coping strategies (Carson, Bittner, Cameron, & Brown, 1994), and better 
leadership abilities (Simonton, 1984). However, creativity is usually considered an end goal in 
itself, and the effect of creativity on other desirable outcomes is not often researched. 

Emotional intelligence. “Is ‘emotional intelligence’ a contradiction in terms?” With that 
question, Salovey and Mayer (1990) started a renewed interest in emotional intelligence (El) as a 
research topic. The idea that emotion is a significant part of our intellectual being has roots in the 
works of Darwin (1872/1965) and Freud (1923/1962), and, more recently, in the work of Cantor 
and Kihlstrom (1985), Gardner (1983), and Sternberg (1985). Cantor and Kihlstrom’s (1985) 
reconception of social intelligence places the ability to solve problems in social situations as a 
core construct of personality. In Gardner’s (1983) theory of multiple intelligences, two of his 
proposed seven intelligences involve emotions: Interpersonal intelligence (understanding other 
people) and intrapersonal intelligence (understanding one’s self). Sternberg’s theory of 
successful intelligence (also known as practical intelligence) is another major theory of intellect 
that takes into consideration the importance of emotional well-being (Sternberg, 1985, 1996; 
Sternberg & Kaufman, 1998). The common historical view, however, has been that emotions are 
secondary—indeed, inferior to—intellect (Salovey, Bedell, Detweiler, & Mayer, 1999). 

Salovey and Mayer’s (1990) initial model of emotional intelligence had three factors: 
appraisal and expression of emotion, regulation of emotion, and utilization of emotion. Appraisal 
and expression of emotion is comprised of emotion in the self (which can be both verbal and 
nonverbal) and emotion in others. Emotion in others consists of nonverbal perception of emotion 
and empathy. The second factor, regulation of emotion, is the ability to regulate emotion in the 
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self, and the ability to regulate and alter emotions in other people. The final factor, utilizing 
emotional intelligence, has four aspects: flexible planning, creative thinking, redirected attention, 
and motivation. Flexible planning refers to the ability to produce a large number of different 
plans for the future, enabling the planner to better respond to opportunities. This production of 
many plans can result from utilization of emotion and mood changes to one’s advantage and 
looking at a wide variety of possibilities. Creative thinking, the second aspect, may be more 
likely to occur if a person is happy and in a good mood. Redirected attention involves the idea 
that when strong emotions are experienced, a person’s resources and attentions may be turned to 
new problems. A person who can use this phenomenon to his or her benefit will be able to use a 
potentially stressful situation to focus on the most important or pressing issues involved. 
Motivating emotions, the final principle of emotional intelligence, refers to the art of making 
one’s self continue to perform difficult tasks by focusing one’s anxiety or tension toward the 
performance of that task. 

This initial model of emotional intelligence has been revised (Mayer & Salovey, 1997; 
Mayer, Salovey, & Caruso, 2000). They now propose four abilities: perceiving, appraising, and 
expressing emotions; accessing and producing feelings in aid of cognition; comprehending 
information on affect and using emotional knowledge; and regulating emotions for growth and 
contentment. 

Since the initial conceptualization of emotional intelligence, there have been some 
alternate definitions presented. Goleman (1995), who popularized the term in his best seller, has 
a broader view of El that includes motivation, self-control, and persistence. Bar-On (1997) 
expanded the concept of El even further, tapping “an array of noncognitive capabilities, 
competencies, and skills” (p. 14) that include assertiveness, independence, empathy, problem 
solving, happiness, and optimism. 

Measurement of emotional intelligence. There are several different methods to measure 
emotional intelligence. These include self-reports, performance tests, ratings, and known groups. 

Self-reports. The Bar-On Emotional Quotient Inventory (EQ-i, 1997) is a self-report 
inventory that has five composite factors: intrapersonal composite (emotional self-awareness, 
assertiveness, self-regard, self-actualization, and independence), interpersonal composite 
(empathy, interpersonal relationship, and social responsibility), adaptability composite (problem 
solving, reality testing, and flexibility), stress management composite (stress tolerance and 
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impulse control), and general mood composite (happiness and optimism). Scores on the EQ-i 
were found to be significantly correlated with self-reported and observer scores on emotional 
intelligence. 

Other self-report measures include the Trait Meta-Mood Scale (Salovey, Mayer, 

Goldman, Turvey, & Palfai, 1995), a predecessor of the MEIS (see below), Emotional Control 
Questionnaire (Roger & Najarian, 1989), Questionnaire Measure of Emotional Empathy 
(Mehrabian & Epstein, 1970), and the EQ test (Goleman, 1995). One criticism of many of the 
self-report tests of El is that they overlap too much with personality tests to provide any new 
information of note (Davies, Stankov, & Roberts, 1998). The EQ-i has been criticized for 
overlapping too much with optimism (J. D. Mayer, personal communication, June 2002) and 
such specific personality tests as the California Personality Inventory (Matthews, Zeidner, & 

Roberts, 2003). In addition, self-report tests have also been criticized for representing a person’s 
perception of their El instead of a person’s actual El (Matthews et ah, 2003)—and, indeed, 
people with low El might not have the insight to recognize this fact and end up rating themselves 
quite high. 

Performance tests. The most recent and most researched perfonnance test of El is the 
Multifactor Emotional Intelligence Scale (MEIS) (Mayer, Caruso, & Salovey, 1999). The MEIS 
consists of 4 branches of El that have three measures each: perceiving, facilitating, 
understanding, and managing emotion. These tests consist of such items as measures of 
emotional perception, measuring judgments of synesthesia (such as “How hot is anger?”), 
understanding of emotion (such as “Optimism most closely combines two emotions,” with 
choices including the correct answer, “pleasure and anticipation”), and scenario tasks involving 
emotional management (Mayer et ah, 2000). The MEIS found that El was correlated with both 
measures of verbal intelligence and self-reported empathy, and that the different measures were 
intercorrelated. A recent large investigation of the MEIS (Roberts, Zeidner, & Matthews, 2001) 
found evidence for convergent and divergent validity but poor reliability. In addition, the two 
scoring processes (expert scoring, in which scores are compared to the response of an expert in 
emotions, and consensus scoring, in which scores are compared to the average response) 
produced different—and often contradictory—results. 

A very recent revision of the MEIS that has not been extensively researched yet is the Mayer- 
Salovey-Caruso Emotional Intelligence Scales (MSCEIT) (Mayer, Salovey, &, Caruso, 2001). 
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Other performance tests include the Emotion Perception Tests (Mayer, DiPaulo, & 
Salovey, 1990), which measures emotion perception in faces, color, music, and sound intervals, 
and the Emotional Accuracy Research Scale (Mayer & Geher, 1996), which consists of 
emotional vignettes and then items that ask the test taker what moods the characters in the 
vignettes are experiencing. The Emotion Perception Tests and other performance tests have been 
criticized for low reliability (Davies et ah, 1998); the MEIS was produced after this study was 
conducted. 

The choice to use a self-report versus a performance assessment may reflect the 
conception of El; a self-report test is similar in format and requirements to personality 
inventories, while performance measures are more comparable to mental ability measures such 
as intelligence tests (Matthews et ah, 2003). One problem with the EARS is that there are two 
types of scores: consensus-agreement, in which the test taker selects the emotion most often 
selected for each vignette; and target-agreement, in which the test taker selects the emotion that 
the real character from the vignette selected. These two scores are not significantly correlated 
(Geher, Warner, & Brown, 2001), which may raise questions about the validity of the 
instrument. 

Ratings. Expert ratings have been used in the assessment of emotional perception, which 
is related to El. Written or videotaped interactions between individuals can be shown to 
participants, who then respond by identifying the emotions the individuals are experiencing 
(Mayer et ah, 2000). These responses are compared to expert responses and then scored by (e.g., 
Ekman, 1999). 

Known groups. Using known groups is not as applicable to the study of emotional 
intelligence because it is not as clear which groups are kn own for high El, as there is no clear 
criteria for selection (unlike creativity, where it would be reasonable to call painters or poets a 
known group). The only way to divide people into high and low emotional intelligence would be 
to measure their El (such as through a performance test). 

Emotional intelligence as predictor. Several proponents of El have argued that it is a 
central determinant of leadership (Caruso, Mayer, & Salovey, 2002; George, 2000; Goleman, 
1995). Several studies have shown a positive relationship between leadership and emotional 
intelligence. For example, higher emotional intelligence in managers as measured by the EQ-i 
was related to increased business success (Dennan, 1999). El accounted for variance in leadership 
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experiences (Kobe, Reiter-Palmon, & Rickers, 2001). And people who were high in El as 
measured by psychometric tests were more likely to be perceived as leaders (Massey, 1999). 

The nature of the relationship between academic and intellectual abilities and El is not 
yet perfectly established. Some studies have used the MEIS and found no relationship with IQ 
(Ciarrochi, Chan & Caputi, 2000); others have found relationships between scores on the MEIS 
and intellectual performance (Lam, 1999; Mayer et ah, 1999). A study of academically 
successful and unsuccessful South African students found that successful students had a 
significantly higher EQ as measured by the EQ-i than the unsuccessful students (Bar-on, 1997). 
However, it is worth noting that only three of composites used in the EQ-i were significantly 
higher (adaptability, general mood, and stress management). 

Predictors of emotional intelligence. Several factors can be predictors of emotional 
intelligence. These include environmental factors, general personality factors, and group factors. 

Environmental factors. Family support and parental influence have been found to have a 
significant positive effect on El as measured by a psychometric test (Martinez-Pons, 1998), 
while a study of breast cancer patients linked social support to El as measured by a test of 
emotional disclosure and a by a cognitive processing task of emotions similar to those found on 
the MEIS (Cunningham, 2000). There has been theoretical speculation (e.g., Ross, 2000) that an 
improvement in the social and emotional climates of the schools would lead to higher student 
emotional intelligence. A different theory, however, argues that female students who are 
academically successful, yet from disadvantaged backgrounds, developed higher levels of 
emotional intelligence (LePage-Lees, 1997). 

General personality factors. Emotional intelligence has been strongly found to relate to 
personality dimensions, particularly emotional stability and agreeableness. Studies that 
administer self-report measures of emotional intelligence and personality inventories (e.g., 

Davies et al., 1998; Murensky, 2000) have found so much overlap with El and personality 
dimensions as to argue that El as measured by self-report inventories does not have sufficient 
divergent validity to distinguish itself from personality measures (see Matthews et ah, 2003, for a 
fuller discussion of the relationship between personality and emotional intelligence). 

Group factors. Goleman (1995) theorized that women would score higher on El, but 
early results do not bear this out. Some studies using the EQ-i have found no differences (e.g., 
Bar-On, 2000), while others using multiple measures (including the MEIS) found conflicting 
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results (e.g., Roberts et al., 2001). The conflicting results showed women scoring higher on the 
MEIS when consensual scoring was used, and men scoring higher when expert scoring was used. 
There is even less research on issues of race and ethnicity, with Matthews et al. (2003) calling 
the available work “scant and contradictory.” Bar-On (2000) reported no differences on the EQ-i. 
Roberts et al. (2001) found no differences with consensual scoring of the MEIS, but found that 
expert scores favored Whites. 

Cognitive styles. Cognitive styles (also thinking styles) are consistencies in how people 
interpret and respond to information or problems presented to them. They are not abilities, but 
rather, characteristic strategies, preferences, and attitudes concerning how to use abilities 
(Messick, 1989; Sternberg, 1997). 

The most well known cognitive style is field dependence-independence (see Witkin & 
Goodenough, 1981 for an overview). Those who rely on the context of a situation, or external 
cues, are called field dependent (or relational); those who rely more on internal cues and focus 
on specific details are called field independent (or analytical) (Anderson & Adams, 1992). A 
criticism of this view is that field independence overlaps with fluid intelligence, and is therefore 
ability rather than a style (Grigorenko & Sternberg, 1995). 

Other cognitive styles include Bruner’s (1986) distinction between paradigmatic and 
narrative thinking. Paradigmatic thought is logical and scientific (what is), while narrative 
thinking is more concerned with seeing connections and telling a story (what might be). 
Sternberg’s (1988, 1997) model of thinking styles, called Mental Self Government, proposes 
three styles: Legislative, Executive, and Judicial. Legislative thinkers prefer to create things and 
to be self-directed. Executive thinkers prefer to follow directions and to work under a great deal 
of structure. Judicial thinkers like to judge and evaluate things. 

Measurement of cognitive styles. Among the different ways to measure cognitive styles 
are self-reports, performance tests, and product ratings. 

Self-reports. There are several self-report measures of cognitive styles; one recent one 
that has been used in several research studies is the Mental Self-Government Thinking Styles 
Inventory (MSG-TSI) (Sternberg & Wagner, 1991). Dai and Feldhusen (1999) found evidence 
for external discriminant validity and partial evidence for internal validity. Several other studies 
(e.g., Wu & Zhang, 1999; Zhang, 1999) used the MSG-TSI in Hong Kong and China and found 
good reliability. 
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Performance tests. Field dependence-independence is typically measured with 
performance tests. The Embedded Figures Test and Group Embedded Figures Test (Witkin, 
Oltman, Raskin, & Karp, 1971) are perceptual tests that show patterns to subjects to gauge their 
ability to distinguish shapes from their field (or background). People who can accurately single 
out different shapes from a variety of backgrounds are field independent. The Rod and Frame 
Test (Witkin, Dyk, Faterson, Goodenough, & Karp, 1962) is an apparatus test requiring 
perceptual judgments. 

Product ratings. Kaufman (2001b) measured Bruner’s (1986) narrative and paradigmatic 
styles by having participants write a sentence in response to an ambiguous photograph and then 
assigning scores for both thinking styles based on Bruner’s definitions. 

Predictors of cognitive styles. Among the factors that can be predictors of cognitive 
styles are domain-specific personality factors and group factors. 

Domain-specific personality factors. There are several studies that suggest cognitive 
styles may be related to interests. Cognitive styles as measured by the MSG-TSI were found to 
distinguish, for example, accounting students (Tucker, 1999), and creative writers vs. journalists 
(Kaufman, 2001b). Field independent people tend to choose more mathematical and scientific 
professions, while field dependent people tend to choose more humanitarian professions (Witkin 
& Goodenough, 1977). 

Group factors. Several studies show no gender differences in cognitive styles (Kaufman, 
2001b; Riding & Rayner, 1998) 

Cognitive styles as predictor. For all the attention they have received, there is 
surprisingly little evidence for the validity of cognitive styles. Sternberg’s legislative style (1988, 
1997), while uncorrelated with IQ and SAT scores (Sternberg & Grigorenko, 1993), has been 
found to predict academic task performance (Grigorenko & Sternberg, 1997). Field 
independence has been shown to correlate with measures of intelligence (Goldstein & Blackman, 
1978), but negatively correlate with extroversion and emotional intelligence (Witkin & 
Goodenough, 1977). However, many of these findings are difficult to evaluate, because studies 
have generally failed to include controls for other ability and personality variables. Perhaps 
because of the lack of any systematic findings, particularly when ability and personality factors 
are controlled for, research on cognitive styles has declined considerably over the years. 
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Metacognition. Metacognition is knowledge about cognition, and metacognitive ability 
is one’s ability to use that knowledge to learn, think, and solve problems more effectively. 
Knowing about and applying mnemonic strategies, such as elaboration and rehearsal, increases 
performance on memory tasks (Kliegl & Philipp, 2002). Knowing and employing effective 
heuristics (e.g., break into subgoals or consider the limiting case) increases problem solving 
performance. Study-skills courses taught in colleges are largely courses to improve 
metacognitive ability. 

A distinction can be made between monitoring and regulating. Regulating refers to the 
selection of methods, such as planning or setting subgoals, for studying or solving problems. 
Monitoring refers to being aware of what one knows or what one has done in a problem solving 
situation (e.g., being aware of whether you know the meaning of a word, or being aware of 
whether you successfully completed a step in a task). Metacognition assessment has tended to 
focus on monitoring, while study-skills courses have tended to focus on regulation (e.g., learning 
strategies, scaffolding). An intriguing finding is that poor performers tend to lack awareness of 
their shortcomings (Ehrlinger & Dunning, 2003; Paulhus, 1998), leading to overconfidence and 
perhaps a tendency to fail to double-check their work, which itself could be the cause of errors. 
This failure to recognize incompetence may partly be due to individuals not being aware of 
errors of omission. 

Measurement of metacognition. Among the different methods to measure metacognition 
are self-reports and perfonnance tests. 

Self-reports. Several authors have developed self-report instruments for general 
metacognitive abilities, but these generally have fared poorly psychometrically (Osborne, 1998; 
Schraw, 2001). This might reflect the domain-specific nature of metacognition. 

Performance tests. Tobias and Everson (1995, 2000) developed a metacognition measure 
called the Knowledge Monitoring Assessment (KMA). It is a methodology for assessing 
metacognition rather than a test per se. It involves administering a test two times—first, to get 
examinees to predict whether they know the answer to each item, one by one; second, to actually 
try to answer the item. The match between the number they predict they will get right, and how 
many they actually get right is the basis for a metacognitive or knowledge monitoring score. The 
method is quite general and could be applied to just about any kind of testing situation. 
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Instead of asking one to predict whether they will get an item correct, one can ask them if 
they got an item correct immediately after they take it, in a kind of reverse KMA (or one can be 
asked to do both, as in the Jr. MAI method of Sperling, Howard, Miller, & Murphy, 2002) This 
is also called a confidence judgment, and it has been used in cognitive psychology for many 
years, often as a score intensifier (examinees get more credit if they solve a problem and are 
confident that their solution was correct [e.g., Lohman, 1980]). Recently Stankov (1999, 2000) 
has used this technique to investigate confidence as an individual-differences construct, finding 
that it is a factor separate from ability per se, a claim often made of metacognition. 

Another method is the Over-Claiming Questionnaire (OCQ) (Paulhus & Bruce, 1990), 
which asks respondents to rate their familiarity with items from a list of general knowledge 
(cultural literacy) items (e.g., Manhattan project), some of which are sound plausible, but 
nonexistent foils (e.g., cholarine). The inclusion of foils allows for the determination of both 
accuracy (the ability to distinguish real items from foils) and bias (the tendency to say yes, even 
to foils). The accuracy measure from this procedure correlates with other ability measures as 
would be expected; the bias measure—which is the metacognition index—correlates with a self¬ 
enhancement personality scale (Paulhus & Harms, in press). 

Predictors of metacognition. There several factors that can be predictors of 
metacognition. These include general personality factors, attitudinal factors, environmental 
factors, and group factors. 

General personality factors. There ought to be a link between personality and 
metacognition because metacognition is an act of reflection on what one is doing, that is, being 
careful and avoiding impulsive behavior, which is a facet of conscientiousness (see the 
cautiousness facet of conscientiousness in Figure 2). However, outside of Paulhus and Harms’ 
finding of a link between self-enhancement and overclaiming, we know of no empirical 
demonstration of this link for either knowledge monitoring or confidence. 

Attitudinal factors. Little work has been done on the relationship between motivation and 
other factors and metacognition; Tobias and Everson (2002) see that as a major research need. 

Environmental factors. Metacognitive strategies can be taught (Palincsar & Brown, 

1987), and in fact, scaffolding methods, which essentially teach metacognitive strategies by 
supporting students through a problem solving attempt, then fading support over time, have 
begun to be routinely incorporated into computerized instructional programs (e.g., Katz & Bauer, 
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2000). However, the generality of such training is not clear. No work to our knowledge has been 
done on the trainability of increasing the accuracy of confidence judgments 

Group factors. Metacognition is a skill that develops with age (Tobias & Everson, 2002). 
Women and men score similarly on metacognition. Little is kn own about racial differences. 

Validity of metacognition. Metacognition as measured by the KMA has been shown to 
predict college reading, college and high school grades, and mathematical achievement (Tobias 
& Everson, 2002). However, the relationships are not strong, and there is an issue of the causal 
direction between metacognition and achievement: studies tend to be correlational. No external 
validity work has been done on confidence judgments. 

Prospects for applied use in admissions decisions. Quasi-cognitive factors seem mostly 
susceptible to coaching and faking. Creativity and emotional intelligence have obvious desired 
outcomes: few schools would specifically seek a student who was not creative or scored low in 
EL Metacognition and cognitive styles do not. But all four factors present problems for use in a 
high-stakes situation. 

Creativity. Creativity measures would be problematic in high-stakes testing. Self-report 
measures could be coached; items tend to be obvious (e.g., “Has a vivid imagination”). 
Biographical data could be falsified or exaggerated (although it could be validated by a referee). 
Existing performance tests could be easily faked—most items depend on large quantities of 
responses rather than high quality, or else they depend on deviation from the norm. A student 
could be trained to fool both types of items. Product ratings are less susceptible, but much of the 
research conducted on these ratings has been in a laboratory setting. When students are given 
specific instructions and are asked to produce essays that are creative in nature, they produce 
work that is rated as more creative (O’Hara & Sternberg, 2001). The effect of such instructions 
(and eventual guides and preparation books) on the validity of ratings has not been studied. One 
kind of creativity measure that could be useful for admissions, would be an essay scored for 
creativity. Whether such a score would differ from a general writing score has not been 
examined. 

Emotional intelligence. El measures fall into two categories: self-reports and 
performance tests. Self-reports, as is true of the personality measures reviewed in a previous 
section, are coachable and fakable. Performance tests may present a possibility. The types of El 
items most like existing cognitive tests, such as those involving vocabulary or concept 
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knowledge, are probably least susceptible to coaching. Some El items have a situational 
judgment test format, in which an examinee’s Likert-scale response is correlated with a reference 
group’s (either the population or a group of experts) to get a score. This item format has proven 
useful for predicting job perfonnance (McDaniel, Morgeson, Finnegan, Campion, & Braverman, 
2001), but whether it is less susceptible to coaching than other item formats has not been studied 
extensively. Coachability may also depend on the reference group (population vs. expert), 
although it is not clear that the responses of one or the other would be easier to anticipate. In 
general, the more cognitive-based the scenario items are, the less likely these items will be 
coachable. 

A further issue with El is whether coaching may affect validity. If an El test asks a 
student to describe what emotion a photographed face is expressing, then this item is trying to 
tap into a student’s abilities to read emotion in other people. If this student has spent three hours 
studying different faces and what the correct label for their emotion is, then this item becomes a 
test of memory—not the originally intended purpose. 

Another concern with El measures focuses on the number of items available. There are a 
limited number of emotion-related vocabulary and concepts. Half of the MEIS (Mayer, Caruso, 

& Salovey, 1999)—judging synesthesia and understanding emotion—uses specific terms related 
to emotion. How many different items could be created to test the ways that a student 
understands emotion? It may indeed be possible to create the quantity of items required by a 
high-stakes test - but this question should be investigated. 

Metacognition, confidence, and cognitive style. Use of these measures in high-stakes 
admissions seems problematic because there is not an obvious desired outcome. Is it better to be 
highly confident or to have more accurate meta-cognition? Is one cognitive style better than 
another? Coaching could influence performance tests for metacognition such as the KMA (e.g., 
instructing a student to say that he or she knows everything); however, for this coaching to be 
effective, there would have to be a clear outcome. Self-report tests of metacognition are less 
effective even without the issue of coaching and faking. 

Thinking styles are also problematic. Field independence may be indistinguishable from 
general cognitive ability. Other styles have no obvious best, but because they are measured with 
self-reports, they are coachable (MSG-TSI) (Sternberg & Wagner, 1991). And most rating 
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measures of thinking styles, such as Bruner’s (Bruner, 1986; Kaufman, 2001b) are designed to 
test a student’s preference, not ability to use a given style. They, too, are undoubtedly coachable. 

Summary. Creativity and emotional intelligence have inspired a great deal of theoretical 
interest, but little research has attempted to evaluate their use in selection or admissions in 
education or elsewhere. Both factors predict achievement and other outcomes, such as 
persistence (creativity), leadership (creativity and El), and business success (El). But it has not 
been demonstrated that such validity would be sustained in a high-stakes context. 

As with other noncognitive factors, including emotional intelligence and creativity 
assessments in admissions would broaden the factors considered, with the additional benefit of 
reducing adverse impact against gender and racial groups. But highly reliable instruments have 
not yet been developed for either factor, nor has sufficient validity data been collected. There 
also may be more domain specificity for creativity and emotional intelligence (see Baer, 1998). 
Creativity may be particularly important for the humanities. Emotional intelligence may be 
particularly important for professions involving analysis and treatment of people, such as 
nursing, clinical psychology, and the like. It is also important to consider that creativity and El 
are often seen as appropriate outcomes in their own right: A creative or emotionally intelligent 
student is a desirable one. Perhaps the ultimate role for these factors will be as criterion outcome 
measures, rather than as predictors of graduate school success. 

Attitudinal Factors 

In this section we review the key attitudinal factors of self-concept, self-efficacy, 
motivation, and interests. All of these factors relate to motivation in that they affect choice of 
activities, goals, strategies, effort, and persistence. They also tend to vary in context: People may 
see themselves as capable, interested, and willing to expend effort in an academic setting, for 
example, but inept, bored, and inclined to give up easily in an athletic one. Although the four 
factors largely reflect different literatures, they overlap conceptually, operationally, and 
empirically, and it seems productive to review them together. 

Why are these factors important to understanding graduate student performance? It has 
been argued that performance depends on declarative knowledge, procedural knowledge, and 
motivation (McCloy, Campbell, & Cudeck, 1994). The GRE primarily reflects declarative and 
procedural knowledge. There is a need to understand attitudes that drive motivation. Further, 
attitudinal factors may provide important infonnation about students who are traditionally 
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underrepresented in graduate programs; students for whom attitudinal factors may be particularly 
important (Zwick, 1991). 

Self-concept. Self-concept refers to “the way people characteristically think about 
themselves.” (Brown, 1998, p. 3), either in general, or in a specific domain. Assessing domain- 
specific self-concept maximizes predictive validity (Vispoel & Wang, 1991). Thus, 
investigations of graduate student experiences focus on academic self-concept, defined as the 
extent to which a person has incorporated academic pursuits into his or her identity. A student’s 
academic self-concept is often a critical factor in educational outcomes and is used as both an 
outcome variable and a predictor of achievement outcomes. 

Related to self-concept is the notion of domain identification. One is identified with a 
domain to the extent that one’s performance within that domain is important to their self- 
concept. Doing well in a domain leads to feeling good about oneself (Steele, 1997), which in turn 
leads to motivation to achieve further in the domain. Students’ academic self-concept and their 
identification with the academic domain affect the value they place on achieving and the 
behaviors they enact. 

Measurement of self-concept. There are several different methods to measure self- 
concept. These include self-reports, domain indentification, biographical data, perfonnance tests, 
ratings, known groups, and observation. 

Self-reports. Self-concept is most often measured with self-reports. A widely used 
general self-concept scale is the Tennessee Self-Concept Scale, Second Edition (TSCS: 2) (Fitts 
& Warren, 1996), which assesses self-concept in the physical, moral, personal, family, social, 
and academic/work areas. It has been used in clinical settings to assess clients' general self¬ 
perceptions to target areas for therapy and remediation. The Self-Description Questionnaire 
(Marsh, Smith, & Barnes, 1983) measures multiple facets of self-concept (Marsh & Shavelson, 
1985) in academic, social, and physical domains, with scales for physical abilities, appearance, 
relationships with peers, relationships with parents, reading, mathematics, and school subjects. 
Subsequent versions of the scale were designed specifically for the late adolescent population 
and high-school students (Marsh & O’Neill, 1984). 

Self-esteem measures, such Rosenberg’s self-esteem scale (1979) can also be used to 
measure self-concept, or specifically, the worth people place on their self-concept. The scale 
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has proven to be a reliable and valid measure of self-esteem for Black students (Hughes & 

Demo, 1989). 

Academic self-concept assessments measure the extent to which a student endorses 
academic attitudes and behaviors. A popular scale is the Dimensions of Self-Concept Scale Form 
H (DOSC) (Michael, Denney, Knapp-Lee, & Michael, 1984), which assesses noncognitive 
factors associated with self-concept and self-esteem in a college setting. The assessment 
comprises several subscales: level of aspiration, anxiety, academic interest and satisfaction, 
leadership and initiative, and identification versus alienation. A sixth factor, stress, was added to 
the scale to account for highly demanding domains (Paik & Michael, 2000). A sample item is, “I 
strive to be one of the best students in every class,” with anchors at never and always. The scale 
was designed to identify students who may have difficulty in their schoolwork due to low self¬ 
esteem, to diagnose perceptions and activities that might contribute to low self-esteem, and to 
identify impaired learning capabilities for purposes of guidance or counseling. The DOSC has 
demonstrated promising concurrent validity with the Maslach Burnout Inventory (MBI) (Gold & 
Michael, 1985) a measure of burnout behaviors relevant to persistence. The DOSC has been 
modified and demonstrated to be valid for Chinese (Huang & Michael, 2000), Spanish 
(Menjares, Michael, & Rueda, 2000), Korean (Chong & Michael, 1999), and Japanese (Paik & 
Michael, 1999) language users. Other scales measure increasingly specific aspects of academic 
self-concept, such as the Mathematical Self-Concept Scale (Gourgey, 1982), which is designed 
to assess attitude toward one's ability to leam mathematics. 

Self-concept scales also assess the extent to which students identify with their ethnic 
identity. Such infonnation can be useful in understanding differences in academic goals, 
strategies and outcomes across and within different groups. Phinney (1992) created the Multi¬ 
group Ethnic Identity Measure (MEIM) to assess three aspects of ethnic identity: Positive ethnic 
attitudes and sense of belonging; ethnic identity achievement, including both exploration and 
resolution of identity issues; and ethnicity-related behaviors or practices. It has been used with 
high school, college, and graduate students. 

Domain identification. Domain identification measures often overlap with both interest 
and academic self-concept inventories. For instance, identification versus alienation is a subscale 
of the Dimension of Self-Concept Scale, which taps students’ level of identification with the 
school setting. Some studies have assessed level of domain identification by considering grades 
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or test scores within an area in conjunction with self-reported attitudes toward that domain 
(Aronson, Lustina, Good, Keough, Steele, & Brown, 1999; Walters, Shepperd, & Brown, 2004). 
For instance, Aronson et al. (1999) selected mathematics students with SAT 8 mathematics 
(SAT-M) scores above 550 to rate the importance of mathematics to their self-concept on a 15- 
point scale with anchors at not at all important to extremely important. Information about domain 
identification can also be gleaned from responses to interest inventories such as the Strong 
Interest Inventory (SII) (Harmon, Hansen, Borgen, & Hammer, 1994). See the Interests section. 

Biographical data. Autobiographical essays, background questionnaires, or even 
interviews could be used to assess self-concept, as themes and repetition could be interpreted to 
reflect the student’s personal identity. Biographical information itself might provide important 
information about a student’s culture of origin, a factor often related to self-concept (Markus & 
Kitiyama, 1991). Myers (1999) suggests asking students to complete the sentence “I am...” with 
five statements, which are interpreted to reflect self-concepts. 

Performance tests. Reaction time measures have been developed to measure implicit 
attitudes and associations, which provides advantages over explicit measures, which are 
susceptible to social desirability responding. The Implicit Association Test (IAT) (Greenwald & 
Banaji, 1995; Greenwald, McGhee, & Schwartz, 1998) measures one’s associations of two target 
concepts with an attribute. Faster response to a concept paired with an attribute implies a 
stronger association between the two. Thus, one might measure an academic self-concept by 
looking at the associations between academic-related concepts and items and the self such as 
homework and self. 

Ratings. Ratings have been used to measure self-esteem, but not self-concept. The 
assumption is that another cannot reliably kn ow the content of a person’s self-concept and the 
degree to which they possess an identity. 

Known groups. To measure academic self-concept, academics could be compared to 
nonacademic professionals (e.g., businesspersons, attorneys) on a variety of other attitude 
measures; but, to our knowledge, this has not been done. Such a comparison might be useful, 
albeit somewhat speculative. 

Observation. A school-based behavioral assessment might identify behaviors indicative 
of an academic self-concept such as class participation, avoiding procrastination, and class 
attendance. Clinicians may seek to understand the content of self-concepts for purposes of 
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identifying goals and possible blocks to achieving those goals. While this infonnation is often 
gathered from conversation, active listening, and observation of academic behaviors, clinicians 
often use self-report measures such as the TSCS; Fitts & Warren, 1996) to assess self-concept. 

In sum, self-report measures are likely the most viable and appropriate way to collect 
information about a person’s self-concept. Other methods might serve to supplement this 
information. However, understanding the self-concept should not involve objective analyses or 
valence judgments. Rather, self-concept is a person’s self-view and is, therefore, best captured 
by the person’s own reports. Interviews with graduate students, open-ended descriptions of their 
graduate experiences, and questionnaires revised for the graduate experiences would be the most 
appropriate means of learning about graduate student academic self-concept and identification 
with the academic environment. 

Predictors of self-concept. Several factors have been shown to predict whether a student 
possesses an academic self-concept. These include such factors as general personality, group, 
and environmental. 

General personality factors. While little research has addressed the relationship between 
a person’s self-concept and domain identification and general personality factors, it is reasonable 
to assume that certain stable factors will influence the formation of one’s self-views, either 
directly or indirectly through others’ reinforcement of particular self-views. For instance, a 
student who is high in conscientiousness may develop productive study habits, which in turn will 
lead to success and identification with the academic domain. Similarly, people’s ratings on 
extroversion, neuroticism, agreeableness, conscientiousness, and openness will likely map onto 
some important component of self-concept. 

Group factors. Black and Hispanic female graduate students in the 1970s consistently 
reported a lower academic self-concept than males (Hurtado, 1994). Within male-dominated 
doctoral programs, women expressed less of an academic self-concept than men (Uelkue-Steiner, 
Kurtz-Costes, & Kinlaw, 2000). Hurtado (1994) implicated low integration among groups of 
graduate students for these differences in self-views. However, others have found that 
incorporating the domain into one’s self-concept depends on how efficacious the person feels in 
that area. Pajares and Miller (1994) found mathematics self-efficacy to mediate the effect of 
gender and prior experience with mathematics on self-concept as well as problem solving 
performance. Men had higher performances and self-concept, but these differences were 
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accounted for by differences in self-efficacy. Gender directly related to prior experiences and 
self-efficacy. Thus, students may identify with a domain to the extent that they view themselves 
as capable in that domain. Among Black graduate students, academic self-concept was found to 
be negatively related to anxiety (Halote & Michael, 1984), implying that feeling anxious about 
one’s ability to do well in an area might impede identification with that area. 

Others have found that what determines an academic self-concept might vary with the 
students’ environments. In an investigation of predictors of academic self-concept among Black 
college students HBCUs and predominantly White institutions (PWIs), Cokley (2000) found that 
the best predictor of academic self-concept for students attending PWIs was grade point average, 
while the best predictor of academic self-concept for students attending HBCUs was quality of 
student/faculty interactions. Additional analyses indicated that grade point average is 
significantly more important for the academic self-concept of Black students attending PWIs 
than for Black students attending HBCUs. Taken together, these studies indicate that self- 
efficacy in the domain is a primary predictor of developing an academic self-concept. However, 
institutional support and perhaps a role model within the domain can be equally important, 
especially among students of color. 

Domain identification is particularly relevant to issues of gender and ethnicity within the 
education environment. It is commonly discussed in terms of group identity; the person views 
their group in relation to that context. Domain identification is often contrasted with feelings of 
alienation within a given domain, or disidentification, defined as “a reconceptualization of the 
self and of one’s values so as to remove the domain as a self-identity” (Steele, 1997, p. 614). 
Disidentification either arises from external influences or from one’s own intentional 
detachment. Crocker and Major (1989) incorporated domain disidentification into their strategies 
of self-esteem maintenance among stigmatized groups. They propose that people often 
disidentify from domains in which their group tends to fare poorly as a means of protecting their 
self-esteem. Hence, Black and Hispanic students in addition to other students whose group bears 
a negative academic stereotype often disidentify from the general academic domain, and women 
disidentify from mathematics and science domains. Steele (1997) argues that limited educational 
prospects for Black students and women, imposed on them by societal structure, frustrates their 
identification with their achievement within the academic domain. In understanding self-concept 
and domain identification as they relate to groups, it is important to consider the group’s culture, 
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history, and relation to the dominant society. For instance, students who grew up in traditionally 
collectivist cultures such as Asia and Latin America will likely have a more interdependent sense 
of self, defined by social roles and relationships with others. Conversely, students from 
individualistic cultures such as the United States will more often develop an independent self- 
concept, defined by individual accomplishments, traits, and goals (Markus & Kitiyama, 1991). 
Ogbu and Stem (2001) describe community forces, the influences of a group’s history of 
socialization, migration, and discrimination, as crucial to group members’ orientation to the 
educational system. 

Environmental factors. Consistent with assumptions that the self-concept is a product of 
one’s experiences with the environment and with other people (Shavelson, Hubner, & Stanton, 
1976), researchers have found a positive relationship between perceptions of faculty support and 
graduate students’ self-concepts (Isangedighi, 1985). A mentor might enhance a student’s 
academic self-concept by modeling academic behavior and by providing support, making the 
academic environment one that is comfortable and thus easy to identify with. This is consistent 
with evidence that faculty support predicts academic self-concept among Black students 
attending HBCUs, but not for those attending PWIs. One might argue that any student who 
makes it to the graduate level will possess a strong academic identity. However, different aspects 
of the environment may more or less fit with the students’ self-concept. For instance, a student 
might be in a doctoral program for counseling psychology because helping people is an 
important part of his self-concept; but, he or she may not be as closely identified to the broader 
academic environment, including research. 

Steele and colleagues (Steele, 1997; Steele & Aronson, 1995) have developed a program 
of research around stereotype threat, a situational perfonnance pressure in which students who 
are highly identified with the academic domain are most susceptible (for more on this topic, see 
the Environmental Influences on Graduate School Outcomes section). The perfonnance pressure 
often leads to suppressed performance in important academic areas, which ultimately can lead to 
disidentification from the domain. Several studies have revealed that modifications to high- 
stakes academic environments can diminish these adverse outcomes. For instance, variations on 
instructions (Steele & Aronson, 1995), ethnic and gender composition of the classroom (Inzlicht 
& Ben-Zeev, 2000), and ethnicity and gender of instructors and test administrators (Walters et 
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al., 2004) have all been found to reduce the situational pressure, and consequently, should reduce 
disidentification from the threatening domains. 

Finally, prior academic experiences predicts self-concept within an academic domain, 
however, the relationship is mediated by self-efficacy (Pajares & Miller, 1994). Thus, the extent 
to which a person identifies with a domain will depend in part on prior experiences, which may 
or may not have created a personal sense of competence within that domain. 

Self-concept as predictor. People select achievement goals that match their conceptions 
of themselves (Swann, 1990). Thus, students might choose to strive for a Ph.D. or another 
advanced degree in part because it reflects that they are high achievers. Tying goals to how we 
think about ourselves (e.g., I’ve always been a top student) increases the value we place on 
attaining that goal (Vallacher & Wegner, 1987), which affects the amount of effort we will 
expend (Oyserman & Markus, 1990). For instance, a positive self-view of succeeding at school 
(e.g., becoming an influential and venerated professor) provides a powerful incentive to work 
hard—especially when contrasted with a possible negative self-image of dropping out of 
graduate school. 

Further, people who see a connection between their goals and a favorable self-concept 
within a domain persist longer at tasks than people who do not see such a connection (Markus & 
Nurius, 1986). For instance, among graduate students in education, self-reported academic self- 
concept was negatively related to reported burnout behaviors as measured by the Maslach 
Burnout Inventory (Gold & Michael, 1985). Among undergraduates majoring in mathematics- 
and science-related fields, only science self-concept emerged as a consistent predictor of second- 
year persistence for men and women (Carlin, 1998). As students progress through graduate 
school, they may reevaluate whether the end goal of attaining the degree continues to match with 
a desirable self-image such as scientist, professor, or lawyer. Moreover, among a wide sampling 
of doctoral students, academic self-concept predicted reported commitment to their chosen field 
(Uelkue-Steiner et al., 2000). 

Self-efficacy. Bandura (1986, 1989) defined self-efficacy as a belief in one’s ability to 
achieve success. Studies have found that self-efficacy is associated with educational success by 
influencing goal selection, intellectual engagement, effort, and persistence. This work has 
involved a wide range of ages and perfonnance domains, and a substantial body of research has 
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focused on self-efficacy as a predictor, mediator and outcome variable associated with success in 
higher education. 

Measurement of self-efficacy. Among the different ways to measure self-efficiency are 
self-reports and performance tests. 

Self-reports. Self-report self-efficacy scales have evolved to encompass a wide variety of 
domains and populations. Scales range from general measures of self-efficacy to highly domain- 
specific scales. General scales include the Self-Efficacy Scale (Sherer, 1982) and the General 
Self-Efficacy Scale (Schwarzer, 1997), which was developed for use in several cultures. 
However, when the goal is to predict outcomes from self-efficacy beliefs, the assessment should 
be congruent with the outcome tasks (Bandura, 1986). For instance, self-efficacy for solving 
mathematics problems is a better predictor of solving mathematics problems than is mathematics 
self-efficacy in mathematics-related courses (Pajares & Miller, 1995). Moreover, global 
academic self-efficacy and domain-specific mathematics self-efficacy are separate, though 
related, latent dimensions (Lent, Brown, & Gore, 1997). 

Therefore, measures of academic self-efficacy should be used to predict academic 
outcomes among graduate students, and even more specific scales can be used to predict 
increasingly narrow aspects of graduate school. For instance, Bandura’s (1989) 

Multidimensional Scale of Perceived Self-Efficacy (MSPSE) includes several subscales, which 
cluster into three general factors: social efficacy, academic efficacy, and self-regulatory efficacy. 
All three factors can apply to specific areas of the graduate domain. 

Several academic self-efficacy scales have been developed for use with undergraduate 
and graduate student populations. A widely used measure of self-efficacy in higher education is 
the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich, Smith, Garcia, & 
McKeachie, 1993). The MSLQ taps into three broad areas related to efficacy: value (intrinsic and 
extrinsic goal orientation, task value), expectancy (control beliefs about learning, self-efficacy), 
and affect (test anxiety). Wood and Locke’s (1987) Academic Self-Efficacy Questionnaire 
(ASE) assesses students’ perceptions of their ability to perform tasks related to higher education. 
The ASE includes subscales that measure both in-class and exam concentration, memorization, 
understanding, explaining and discriminating among concepts, and note taking. Also, the College 
Academic Self-Efficacy Scale measures perceptions of academic ability among college students 
(Owen & Froman, 1988). 


51 



Other self-efficacy scales measure increasingly specific aspects of the academic 
environment. For instance, the Mathematics Self-Efficacy Scale (MSES), Forms A and B, 
measures beliefs about one’s ability to perfonn various mathematics-related tasks and behaviors 
(Betz & Hackett, 1983). The Computer Self-Efficacy Scale assesses perceptions of computer- 
related knowledge and skills (Murphy, Coover, & Owen, 1989). The Career Decision-Making 
Self-Efficacy Scale (CDMSE) measures students’ self-efficacy in their ability to make career 
decisions (Taylor & Betz, 1983). Some scales have been designed specifically for graduate 
students. For instance, the Research Training Environment Scale-Revised (RTES-R) is a 
measure of research interest and self-efficacy among counseling graduate students (Kahn & 
Gelso, 1997). Kahn and Miller (2000) developed the RTES-R-S, a shortened form of the RTES, 
which demonstrated adequate internal consistency and corresponded closely to total scores from 
the original RTES-R. Also, the Self-Efficacy Towards Teaching Inventory-Adapted measures 
efficacy among graduate student teaching assistants (Prieto & Altmaier, 1994). The Counselor 
Self-Efficacy Scale (CSES) was designed to assess skills and confidence in counseling activities 
for counseling graduate students (Baker, 1989). The scale first assesses whether the student can 
perform a behavior, and second how confident the student is in performing that behavior. 

Other inventories address experiences of specific groups. For instance, Solberg, O’Brien, 
Villareal, and Davis (1993) developed an assessment for Hispanic students’ self-efficacy for 
adjusting to the college environment. Self-Efficacy Expectations for Role Management 
(SEERM) assesses women’s perceptions of efficacy for managing multiple roles such as parent 
and spouse/partner in addition to worker or student (Lefcourt, 1996). 

Performance tests. Students’ responses to scenario items may also reveal feelings of self- 
efficacy. Considering that students are more likely to avoid tasks in which they feel low self- 
efficacy and to approach tasks in which they feel high in self-efficacy (Bandura, 1986), students’ 
decisions to approach a hypothetical performance task may provide information about their self- 
efficacy beliefs within that domain. 

Predictors of self-efficiency. Several factors can be predictors of self efficiency. These 
include such factors as group, environmental, and personality. 

Group factors. Students of color generally have lower graduation rates than do their 
White counterparts in undergraduate (Bowen & Bok, 1998) and graduate programs (Zwick, 
1991), and several studies suggest that self-efficacy may play an important role in these group 
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differences. Self-efficacy is a primary predictor of academic persistence among Black 
undergraduate students attending PWIs (Steward & Jackson, 1990; Tracey & Sedlacek, 1984). 
Further, self-efficacy predicted academic performance among Black undergraduate students at 
HBCUs (Wilson, 1997). However, Black female students who reported having encountered 
negative racial attitudes reported lower academic self-efficacy (Byars, 1998). Taken together, 
this research suggests that Black students who report high self-efficacy will have greater 
academic success and persistence, but this efficacy can be undermined by experiences with 
negative group stereotypes. 

Self-efficacy is a particularly important issue for women pursuing fields in which they 
have traditionally been underrepresented such as science, mathematics, and engineering. An 
investigation of women graduate students for the National Academy of Sciences found a lack of 
adequate preparation for women entering quantitatively oriented programs and diminished self- 
efficacy (Dix, 1987). However, among science and engineering graduate students, expectations 
about student/faculty interactions were a more significant predictor than was gender for 
academic self-efficacy (Santiago & Einarson, 1998). Thus, supportive faculty members might 
enhance efficacy among women in quantitative fields. Further, Pajares and Miller (1994) found 
mathematics self-efficacy to be more predictive of mathematics achievement (as measured by 
problem solving) than was gender or prior experiences. Self-efficacy also mediated the effect of 
gender and prior experiences on mathematics self-concept and problem solving. This research 
suggests that enhancing self-efficacy may be pivotal for maximizing performances for women in 
quantitative fields. 

Self-efficacy also influences the academic outcomes of nontraditional students, students 
who are older than average students, and those who have multiple life roles. Undergraduate 
nontraditional students (aged 25 years and older) showed the strongest positive correlation 
between academic self-efficacy and GPA (Tiner, 1995). Sandler (1999) found that the Career 
Decision-Making Self-Efficacy Scale (CDMSE) of Taylor and Betz (1983) predicted persistence 
among nontraditional doctoral students (typically defined as aged 30 years and older), indicating 
that confidence in the career choice may be an incentive to persist in training for the chosen 
career path. However, multiple role conflict negatively related to academic self-efficacy among 
graduate students (in educational research courses) (Feldmann & Martinez-Pons, 1995). Thus, 
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self-efficacy is highly important to outcomes of nontraditional students, but these feelings may 
be lessened by excessive external demands such as family and professional roles. 

Environmental factors. Among graduate science and engineering students, student 
perceptions of preparedness and student-faculty interactions were strong predictors of academic 
self-efficacy and career-related outcome expectations (Santiago & Einarson, 1998). Among 
doctoral students from the College of Education, research self-efficacy was best predicted by 
perceptions of a supportive relationship with their advisor (Faghihi, 1998). Also, education 
students who held graduate assistantships reported more research self-efficacy than did students 
without assistantships (Faghihi, 1998). For doctoral candidates, research self-efficacy was an 
important factor in academic goal achievement and degree attainment. Students often reported 
high efficacy in the literature review of their dissertation, but low efficacy in the research design 
and analysis component of their research. Performance in statistics independently accounted for 
a high degree of the variance in self-efficacy (Bako-Okolo, 1996). Thus, doctoral students who 
do not feel competent in the technical and research aspects of their dissertation may be least 
likely to complete their dissertation. 

Personality factors. Self-efficacy is often a key component to research on interests and 
personality. People may develop an interest in areas in which they feel that their likelihood of 
success is high (Lent, Brown, & Hackett, 1994). For instance, vocational interest was a strong 
predictor of academic self-efficacy among students in engineering and science (Hackett, Betz, 
Casas, & Rocha-Singh, 1992). Scientist interest predicted research self-efficacy among doctoral 
students in counseling psychology (Geisler, 1996). Among graduate students in counseling 
psychology, research self-efficacy was positively correlated with positive attitudes toward the 
research-training environment (Phillips & Russell, 1994). 

Self-efficacy scales have begun to be used in conjunction with interest measures in 
vocational counseling. Self-efficacy in an area of interest is considered to be an important aspect 
of vocational decision-making. For instance, the Strong Interest Inventory can be supplemented 
by the Skills Confidence Inventory (SCI) (Betz, Borgen, & Hannon, 1996). The SCI consists of 
ten activities, termed general confidence themes, relevant to the six Holland (1959, 1973) types. 
Respondents rate the extent to which they feel confident in their ability to complete each activity. 
An interest/self-efficacy pattern emerges as part of the overall strong interest profile. 
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Self-efficacy has also been shown to relate to specific personality factors. For instance, 
Carver and Scheier (1982a) found that people who are high in private self-consciousness are 
more likely to persist on a task only when they feel efficacious in the task domain. Among 
privately self-conscious people, self-efficacy predicts superior perfonnances (Brockner, 1979; 
Carver & Scheier, 1982b). Further, optimism significantly predicted self-efficacy among 
undergraduate students (Chemers et al., 2001). Moreover, Judge, Locke, and Durham (1997) 
proposed that emotional stability (low neuroticism), generalized self-efficacy, locus of control, 
and self-esteem comprised core self-evaluations, and a higher order construct. In a meta-analysis 
of the relationships among these variables, generalized self-efficacy and emotional stability 
showed a strong positive correlation (Judge & Bono, 2001). However, in an analysis of 
responses to Eysenck’s Personality Questionnaire, efficacy measures were unrelated to 
introversion, neuroticism, and psychoticism (Eysenck & Eysenck, 1991). 

Self-efficacy as predictor. Self-efficacy exerts a strong influence on academic 
performance through its influence on the goals people select and the behaviors they enact toward 
those goals. For instance, people select academic goals in which they believe that there is a high 
likelihood of succeeding. In an analysis of undergraduates, high self-efficacy (as measured by 
the Academic Self-Efficacy Scale, Wood, & Locke, 1987) predicted high personal academic 
performance goals (i.e., grades), and performance goals and ability were related to actual course 
performance. People who are high in self-efficacy are also more likely to adopt goals related to 
acquiring knowledge (learning goals) as opposed to simply performing well (perfonnance goals) 
(Greene & Miller, 1996) (for more on this, see the Motivation section). Zimmerman and Bandura 
(1994) found perceived academic self-efficacy to influence writing grade attainments for college 
freshman by fostering the adoption of learning goals for writing skills. Goal selection, then, 
affects performance outcomes. Greene and Miller (1996) found that self-efficacy and learning 
goals predicted meaningful cognitive engagement, which in turn influenced achievement. 

Self-efficacy is also apparent in the strategies that people adopt to reach their academic 
goals. For instance, people who have high self-efficacy adopt more efficient and sophisticated 
problem solving strategies (Bandura & Wood, 1989), and develop more effective time 
management skills (Moore, 1995). People with high self-efficacy also seek more infonnation and 
spend more time practicing toward achieving their goals than do people with low self-efficacy. 
They also work harder and persist longer at tasks, particularly in the face of obstacles (Bandura, 
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1986, 1989). Numerous studies have demonstrated the affect of self-efficacy on college and 
graduate school performance. Among doctoral students from the College of Education, research 
self-efficacy contributed to dissertation progress (Faghihi, 1998). Among counseling doctoral 
students, research self-efficacy predicted research productivity and science-related career goals 
(Kahn & Scott, 1997). Self-efficacy was the most influential predictor of dissertation progress 
among doctoral students in counseling psychology programs (Geisler, 1996), and the strongest 
predictor of academic achievement among engineering and science undergraduates (Hackett et 
ah, 1992). Among college freshman, academic self-efficacy and personal goals predicted exam 
performance (Vrugt, Hoogstraten, & Langereis, 1997). In an analysis of a wide range of possible 
predictors, self-efficacy contributed significant unique variance to the prediction of grades and 
persistence among undergraduates considering technical and scientific fields (Lent, Brown, & 
Larkin, 1986). 

Students’ perceived self-efficacy has been found to be highly predictive of academic 
outcomes within a wide range of educational domains. For instance, self-efficacy predicted 
performance in reading comprehension (Schunk & Rice, 1993; Shell, Colvin, & Bruning, 1995), 
writing (Graham & Harris, 1989; Pajares & Johnson, 1996; Schunk & Swartz, 1993a, 1993b; 
Shell et ah, 1995), mathematics (Pajares & Kranzler, 1995; Pajares & Miller, 1994, 1995), and 
performance on a computerized, high-stakes certification examination for male Certified 
Netware Engineer TM candidates (Mulkey, 1997). Perceived self-efficacy mediates perfonnance 
in these areas, in large part, due to its influence on choice of activities, goal setting, strategy 
selection, task persistence, and help-seeking behaviors (Bandura, 1986; Schunk, 1991; 
Zimmerman & Martinez-Pons, 1990). 

Generalized self-efficacy was one of four components of Judge et al.’s (1997) core self- 
evaluation construct, which is significantly related to job satisfaction and performance (Judge & 
Bono, 2001; Judge, Locke, Durham, & Kluger (1998). Further, both domain-specific self- 
efficacy (Stajkovic & Luthans, 1998) and generalized self-efficacy (Judge & Bono, 2001) are 
independently positively related to both job satisfaction and job perfonnance. 

Motivation. “There are three things to remember about education,” former Secretary of 
Education Terrel Bell said. “The first is motivation. The second one is motivation. The third one 
is motivation” (Maehr & Meyer, 1997; cited in Covington, 2000, p. 171). 
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Current theories of motivation are concerned with beliefs and cognitions (Eccles, 
Wigfield, & Schiefele, 1998), seeing motivation as “the decision to act, the intensity of the 
action, and the persistence of action” (McCloy et ah, 1994). A key distinction in the motivation 
domain is between intrinsic and extrinsic motivation. Intrinsic motivation is performing an 
activity for enjoyment. Extrinsic motivation is performing an activity for an external reason (e.g., 
a reward or public recognition) (Deci & Ryan, 1980, 1985). One conceptualization of intrinsic 
motivation is Csikszentmihalyi’s (1988, 1990) conception of flow. Flow is an optimal 
experience—defined as the sensations and feelings that come when an individual is actively 
engaging in an intense, favorite pursuit. This could be anything from rock climbing to playing 
the piano. An individual’s abilities must match the potential challenges of the situation to enter 
the flow state—someone who never played the piano would not consider trying to master 
Beethoven as a flow state; and a concert pianist would not enter the flow state trying to play 
“Mary Had a Little Lamb” (Csikszentmihalyi, 1990). 

Another intrinsic motivation idea is effective motivation. If an individual succeeds at a 
task and is rewarded, then the reward is internalized, increasing intrinsic motivation (Harter, 
1981). Another idea is self-detennination (Deci & Ryan, 1985), which is that because of our 
need for competence, we prefer optimal challenges to increase skills. Intrinsic motivation occurs 
when we are both interested in the activity and self-detennined. 

Motivation can depend on our goals. Perfonnance goals can be distinguished from 
learning goals (Nicholls, 1979). When the goal is performance then the way we respond to 
difficult tasks depends on our perceived ability. With a high level of perceived ability, 
difficulties are handled with an effective problem solving approach, referred to as performance- 
approach. With a low level of perceived ability, there is a learned-helpless response, referred to 
as performance-avoidance (Elliot & Church, 1997; Middleton & Midgely, 1997). In contrast, 
when the goal is learning, there is a mastery orientation response to difficulty regardless of the 
perceived level of ability (Elliott & Dweck, 1988). Zwahr-Castro (2000) found that goals served 
as a mediator for motivation, with mastery goals mediating achievement needs and intrinsic 
motivation and both perfonnance-avoidance and perfonnance-approach goals mediating fears of 
failure and intrinsic motivation. 
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Measurement of motivation. Several methods can be used to measure motivation: self- 
reports, biographical data, performance tests, ratings/product ratings, known groups, and 
observation. 

Self-reports. One direct measure of intrinsic and extrinsic motivation is Amabile’s (1995) 
Work Preference Inventory (WPI) . The WPI is a set of thirty self-report items that are rated 
never true, sometimes true, often true, and always true. These items include “I am keenly aware 
of the GPA goals I have for myself’ and “What matters most to me is enjoying what I do.” These 
items produce an intrinsic motivation and an extrinsic motivation score. The WPI has been found 
to have internal consistency, test-retest reliability, and is correlated with other behavioral 
measures of motivation (Amabile, Hill, Hennessey, & Tighe, 1994). Other self-report measures 
that are related to the construct of intrinsic motivation include the Academic Motivation Scale 
(see Cokley, 2000), Achievement Goals Questionnaire (Elliot & Church, 1997), the Situational 
Motivational Scale (see Guay, Vallerand, & Blanchard, 2000), and the Student Interest and 
Experience Questionnaire (Amabile, 1989). 

Biographical data. Biographical data is not often used in empirical studies of motivation, 
although there is a suggestion that it could be used to make judgments about an individual’s 
motivation (Brown & Campion, 1994). 

Performance tests. While the authors of the School Motivation Analysis Test (Sweney, 
Cattell, & Krug, 1970) refer to it as an objective (i.e., perfonnance) test, the methodology places 
it closer to self-report (although the scoring system claimed to eliminate scorer bias and social 
desirability). 

Ratings/product ratings. Another method of assessing a student’s motivation is to ask his 
or her teacher to give a rating of motivation. One standardized method of getting these ratings is 
to use the Teacher Rating of Academic Achievement Motivation (TRAAM) (Stinnett, Oehler- 
Stinnett, & Stout, 1991). The TRAAM has been shown to have construct validity with self-report 
measures of motivation (Stinnett & Oehler-Stinnett, 1992) and to have higher predictive validity 
for SAT scores than other self-report measures (Schuck, Oehler-Stinnett, & Stout, 1995). 

Known groups. The use of known groups is not as commonly used in the assessment of 
motivation due to the lack of clear criteria for selection; people can only easily be divided up into 
groups based on motivational orientation if they are already tested or previously observed. 
However, known groups can be used in experimental studies that offer an external reward to one 
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group of participants—thereby, according to Deci and Ryan (1985), making these participants 
more extrinsically motivated—and not offering an external reward to a different group of 
participants. 

Observation. There are two ways that observation plays a role in the assessment of 
motivation. One way is to present participants with a task that is presumed to be inherently 
enjoyable, such as drawing pictures or solving puzzles (e.g., Lepper, Greene, & Nisbett, 1973) 
and then offer them an extrinsic reward for completing the task. Participants are then observed 
when they are not being rewarded to see if they still choose to engage in the tasks. Participants 
who choose to spend time on the tasks even when no reward is present are presumed to be more 
intrinsically motivated to engage in the task, while those participants who do not spend time on 
the tasks are presumed to be more extrinsically motivated. 

A second way to use observation in the assessment of motivation is to observe teachers 
and students in a classroom setting and then drawing conclusions about motivational factors such 
as goals and effort is another method that has been used to assess motivation (Barlia, 1999; 
Brookhart & DeVoge, 1999). Usually, these observations are conducted in conjunction with 
other assessments, such as self-report inventories. 

While these traditional measures are the most common way of assessing a student’s 
motivation, it is important to note that these measures do not take into account the broader 
context in which a student’s motivation may be present. Students who are members of minority 
groups, for example, may be influenced by a wide array of factors that pertain specifically to 
their social group (such as societal rewards and pressures), and these factors may not reflected in 
commonly used measures (see Ogbu, 2001). 

Predictors of motivation. There have been few studies that have directed examined the 
relationship of intrinsic versus extrinsic motivation and the Big 5 personality factors. One study 
that administered the WPI and the NEO-PI to a sample of college students found that openness to 
experience was significantly positive correlated with intrinsic motivation, while openness to 
experience and agreeableness were significantly negatively correlated with extrinsic motivation 
(Kaufman, 2001b). Learning styles associated with achievement motivation (meaning-directed, 
reproduction-directed, and application-directed) were also found to be correlated positively with 
conscientiousness, extraversion, openness to experience, and agreeableness and to be correlated 
negatively with neuroticism (Busato et ah, 1999). Extraversion was also found, however, to be 
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associated with sensitivity to signals of reward and generalized reward expectancy, two aspects 
of extrinsic motivation (Zuckerman, Joireman, Kraft, & Kuhlman, 1999). A study of intrinsic 
and extrinsic career success—measuring such variables as job satisfaction for intrinsic, and 
salaries and job level for extrinsic—found that extroversion was positively correlated with 
intrinsic success, neuroticism and conscientiousness were negatively correlated with intrinsic 
success, and agreeableness was negatively correlated with extrinsic success (Boudreau, Boswell, 
& Judge, 2001). 

Motivation as predictor. Mastery goals lead to mastery orientations and intrinsic 
motivation; this combination leads to better graded perfonnance (Church, Elliott, & Gable, 
2001). Perfonnance goals lead to negative affect and decreased working memory, while mastery 
goals lead to positive affect and increased working memory (Linnenbrink, Ryan, & Pintrich, 
1999). Perfonnance goals are less negatively associated with these outcomes with male students 
and with older students (Midgely, Kaplan, & Middleton, 2001). 

Sometimes, goals have an indirect effect on achievement. Among undergraduates, 
perceived ability (self-efficacy) and mastery learning goals influenced meaningful cognitive 
engagement, which in turn influenced achievement. Performance goals predicted shallow 
processing which negatively influenced achievement (Greene & Miller, 1996). In addition, 
students with mastery goal orientations used more effective strategies, preferred challenging 
tasks, had a more positive attitude toward the class, and a stronger belief that success follows 
from effort. Perfonnance goals were related to a focus on ability, and evaluating their ability 
negatively (Ames & Archer, 1988). 

Intrinsic motivation has long been associated with task persistence. Indeed, many of the 
central studies that defined the construct of intrinsic motivation (e.g., Lepper et al., 1973) used 
persistence in a task as the criteria for whether an individual was intrinsically motivated—if an 
individual persisted in solving a puzzle or playing a game, then he or she was considered to 
enjoy doing the task, and was therefore considered to be intrinsically motivated. Several other 
studies (e.g., Reeve & Nix, 1997; Seth, 1996) have included a measure of persistence as an 
additional measure in a larger study, and these studies have consistently found intrinsic 
motivation to be related to higher levels of persistence. 

Structured interviews with 22 graduate students revealed that they felt there was more 
structure and direction associated with coursework than with the independent activity required to 
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complete a dissertation. Thus, a student needs to be highly self-motivated and self-directed to 
successfully complete their degrees (Kluever & Green, 1998). Culture is also an important aspect 
to consider here, however, as high self-motivation may not manifest itself in school performance 
in all cultures (Suarez-Orozco & De Vos, 1994). 

There is a large literature on the impact of motivation on creativity, much of it focused on 
the distinction between intrinsic and extrinsic motivation. The research of Amabile and her 
colleagues (Amabile, 1979, 1982, 1996; Amabile & Gitomer, 1984; Amabile et ah, 1986; 
Amabile et ah, 1994) argued that intrinsic motivation (perfonning an activity out of enjoyment 
for that activity) is more conducive to producing creative work than extrinsic motivation 
(perfonning an activity for an external reason, such as a reward; see Deci & Ryan, 1980, 1985; 
Lepper et ah, 1973). 

Amabile (1985) studied the effects of an intrinsic versus extrinsic motivational 
orientation on creative-writing graduate and undergraduate students. Participants in this study 
first had to write a poem to establish a baseline of creative writing. She then gave them a list of 
reasons for writing. One group received lists that stressed extrinsic motivation (i.e., “You want 
your writing teachers to be favorably impressed with your writing talent,” “You know that many 
of the best jobs available require good writing skills”), while another group received lists that 
emphasized intrinsic motivation (i.e., “You enjoy the opportunity for self expression,” “You like 
to play with words”). Amabile had the students rank-order these reasons, and then write a second 
poem. Outside raters evaluated both poems. The students who were given the list of intrinsic 
reasons to rank, as well as a control group that received no lists, showed no significant difference 
in the ratings of creativity. The students given the extrinsic list, however, were rated significantly 
lower on their second poem. 

Ruscio et al. (1998) examined which task behaviors best predicted creativity in three 
domains (problem solving, art, and writing). The most important indicator was found to be a 
participant’s involvement in the task, as measured through behavioral coding and think-aloud 
protocol analysis. Other predicting factors differed by domain. In the domain of writing, which 
was measured with a Haiku poem-writing task, the other central indicator of creativity was a 
factor called striving. Striving was comprised of difficulty, transitions, questioning how to do 
something, repeating something, and positive and negative exclamations. 
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In addition, Amabile et al. (1986) looked at the effect of reward on student creative 
performance. They had a reward and a no reward condition, and then had a second condition as 
to how the task was presented—work, play, or no label. In the reward condition, the children 
were offered the use of a Polaroid camera, a desirable activity for these children, if they would 
promise to tell a story later. In the no-reward condition, the children were also allowed the use of 
a Polaroid camera, but this was presented as merely another task, not as a reward for future 
activity. After the children in all conditions took photographs, they were then asked to tell a 
story, based on a picture book. In the work condition, the story-telling task was labeled work, 
while in the play task it was labeled play. The no-label condition did not use a label for the 
storytelling activity. These stories were then judged by outside raters. Amabile et al. (1986) 
found that children told more creative stories if they were in the no-reward condition, while no 
significant effect was found for the task labeling condition. 

The harm to creativity from rewards may be minimized. Students who received intrinsic 
motivation training (such as directed discussion sessions that focused on intrinsic reasons for 
performing the task in question) before performing a task and receiving a reward showed less 
negative effect (Hennessey, Amabile, & Martinage, 1989). Yet Cooper, Clasen, Silva-Jalonen, 
and Butler (1999) found that even with set in an intrinsically motivating context, rewards had a 
negative impact on performance. 

Although extrinsic motivation can impair creativity, intrinsic motivation can enhance 
creativity. Greer and Levine (1991) found that students given an intrinsic motivation introduction 
wrote poems that were judged to be more creative than those produced by a control group. 

Recently, however, some reviews of the motivation research have challenged the 
assertion that intrinsic motivation is linked to higher perfonnance (and increased creativity). 
Cameron and Pierce (1994) conducted a meta-analysis of 96 experimental studies involving the 
effects of reward on intrinsic motivation, and found that the only negative effect came from a 
reward being tangible, expected, and given for the performance of a simple task. Eisenberger and 
Cameron (1996) argued that rewards (which result in extrinsic motivation) are not necessarily 
detrimental to perfonnance. They stated that the detrimental effects occur under restricted and 
avoidable conditions and that reward can often have a positive effect on creativity. Further 
studies have shown that intrinsic motivation and creativity were not negatively affected by a 
reward (particularly a verbal reward), and could actually be improved, if the reward was 
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presented in a less salient manner, especially in tasks requiring divergent thinking (Eisenberger 
& Selbst, 1994). Eisenberger, Armeli, and Pretz (1998) found that a promised reward increased 
creativity if students received training in divergent thinking, or if instructions emphasized the 
need for creativity, while Eisenberger, Haskins, and Gambleton (1999) found that rewards 
increased creativity if the students had prior experience with creative acts. 

There has been extensive research on the potential benefits of intrinsic motivation and the 
potential harm of extrinsic motivation; while it certainly appears that there is an association 
between increased intrinsic motivation, decreased extrinsic motivation, and higher levels of 
creativity, the nature and extent of these associations has not yet been fully determined. 

Motivation may depend on a student’s culture of origin. For example, the intrinsic- 
extrinsic distinction might be particularly important in individualistic cultures, which emphasize 
the value of achieving for one’s own personal satisfaction. Individuals from collectivist cultures 
value group connectedness and may be more motivated by variables such as group harmony (De 
Vos & Suarez-Orozco, 1990; Markus & Kitiyama, 1991). Free choice is an important variable in 
increasing intrinsic motivation. Recent work has shown that personal choice is less important for 
an Asian child’s motivation that an American child’s motivation (Iyengar & Lepper 1999). 

In a related vein, researchers have noted that achievement motivation may depend on 
whether a group’s migration to the United States was voluntary as opposed to involuntary (Ogbu, 
2001; Ogbu & Simons, 1998). For example, traditional voluntary migrants were able to leam that 
academic achievement is the key to upward social mobility and were able to quickly adopt this 
value as their own. In contrast, nonvoluntary migrants may not have experienced such a 
continuous link between education and success. 

The empirical picture is mixed. A number of studies have found no gender differences in 
intrinsic or achievement motivation in samples of returning college students (Sachs, 2001), 

White and Black junior high school students (Forkner, 2001), Asian American and White 
medical students (Krishnan & Sweeney, 1998), and Hispanic junior high students (Badiozamani, 
1995). Other studies have found no race/ethnic differences in motivation between Black and 
White students (Forkner, 2001), Asian and White students (Chun, 2000), and Irish and American 
business students (De Pillis, 1999). The impact of intrinsic motivation may vary with gender and 
race; women are more likely to produce less creative work, for example, under extrinsic 
constraints (Baer, 1997), and male students have been found to have lower verbal motivation and 
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higher mathematics motivation than female students (Skaalvik & Rankin, 1994). As mentioned, 
Anglo American students showed less intrinsic motivation when decisions were made for them 
by other people, while Asian American students showed more intrinsic motivation (Iyengar & 
Lepper, 1999). 

Attributions. Attributions concern the nature of the source to which one attributes 
successes and failures. For example, after receiving a good grade on an essay assignment, a 
student might attribute that grade to either her own hard work (internal) or to good instruction 
(external). The student might also believe that she can repeat the success because she is a good 
essay writer (stable) or that sometimes she’s on and sometimes off when it comes to essay 
writing (unstable). The student might believe the performance to be something she can maintain 
as long as she works hard and concentrates (controllable), or that essay writing is a matter of 
inspiration, which might or might not be present when the assignment is due (uncontrollable). 
The student could believe that her essay writing skills are general to any topic (global) or that 
they are domain specific—that she could be a good essay writer in political theory, but a poor 
one in cosmology (local). These conceptions stem from Rotter’s (1966) work on locus of control 
(internal vs. external), later extended by Weiner (1985) and Seligman and colleagues (Abramson, 
Seligman, & Teasdale, 1978; Seligman & Schulman, 1986), and are referred to variously as 
explanatory style and attributional style. 

Measurement of attributions. Among the ways to measure attributions are self-reports 
and perfonnance tests. 

Self-reports. The most widely used instrument is Peterson, Semmel, von Baeyer, 
Abramson, Metalsky, and Seligman’s (1982) Attribution Style Questionnaire (ASQ), which 
consists of 6 positive and 6 negative hypothetical events to which the examinee detennines a 
likely attribution from among several alternatives. Alternatives vary along the dimensions of 
intemality, stability, and globality. 

Because the dimensionality of attributional styles is well researched and fairly 
straightforward, it is not too difficult for an investigator to develop an attributional style 
questionnaire for specific domains, or for specific research purposes. For example, Emmerich, 
Rock, & Trapani (2004) developed the Teacher Attribution Inventory—Short Form (TAI-S) to 
measure characteristic attribution patterns for teachers in a study of detenninants of teaching 
success. The instrument assesses causal attributions for a successful and for an unsuccessful 
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teaching event. It consists of explanations varying in internality, stability, and controllability 
(internal-stable-controllable, external-unstable-uncontrollable, etc.), and also strong vs. weak 
susceptibility to self-enhancement. 

Performance tests. The content analysis of verbatim explanations (CAVE) technique 
involves analyzing speeches or writings of an individual to assess attribution style. The method 
has been applied to studying politicians from a distance (Zullow, Oettingen, Peterson, & 
Seligman, 1988), as well as students and others in a more conventional assessment setting 
(Schulman, Castellon, & Seligman, 1989). CAVE assessments are comparable to (i.e., highly 
correlated with) ones made with self-reports (Shulman et ah, 1989). 

Attributional style as predictor. An adaptive attributional style is one that increases task 
effort and persistence. What is adaptive depends on whether the attributions follow a success or 
failure experience (Graham & Weiner, 1996). Following a success, an adaptive attributional style 
is one in which the cause of the success is seen as internal (something about me), stable (it will 
stay this way), and controllable (something I made happen). Following a failure, an adaptive 
style is one in which the cause of the failure is seen as internal, unstable (I can change it), and 
controllable. 

A line of research that has demonstrated the importance of adaptive attributions is the 
studies of Dweck and colleagues of the effects of thinking of intelligence as a fixed entity vs. 
intelligence as a modifiable characteristic (Dweck & Leggett, 1988; Dweck & Sorich, 1999; 

Licht & Dweck, 1984). This research investigates students’ responses to academic obstacles and 
setbacks. They found some students responded with renewed enthusiasm to obstacles, while 
others responded with helplessness and resignation. They related this to the views the students’ 
held about the nature of intelligence. Students who view intelligence as malleable view obstacles 
as challenges, in a mastery-oriented coping style. They see hard work as the key to achieving 
success. Students who believe that intelligence is a fixed entity respond with helplessness to 
obstacles, and believe that no amount of effort can improve their station. These differences in 
theories about intelligence can have a profound effect on academic achievement through their 
effect on the goals a student adopts. Students who hold entity beliefs about intelligence adopt 
performance goals, wherein their goal is to secure favorable views from others via good grades 
and praise. Students with a mastery orientation adopt learning goals, in which their objective is 
to increase competence. 
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Interests. Lent, Brown, and Hackett (1994) define vocational interests as “patterns of 
likes, dislikes, and indifferences regarding career-relevant activities and occupations” (p. 80). 
Like most explanations of interests in the literature, this definition is operational rather than 
conceptual. As a consequence, most studies have focused heavily on the development of 
psychometric scales to predict career-related outcomes as opposed to theoretical developments of 
interest as a construct (Savickas, 1999). Extensive research has proven the usefulness of interest 
inventories in predicting important vocational and academic outcomes. 

Research on vocational interests has proliferated over the past seven decades. Studies 
have typically looked at the match between a person and a job, and the relation of that fit to 
outcomes such as job performance and satisfaction. Holland’s (1959, 1973) model of vocational 
interests has provided an important framework for assessing this person-environment fit. He 
proposed that interests map onto six personality types: realistic, artistic, investigative, social, 
enterprising, and conventional. Each type relates to environments that promote certain attitudes 
and abilities. For instance, realistic types prefer environments that capitalize on realistic skills 
such as mechanical abilities; investigative types like investigative endeavors such as 
mathematics and science. Artistic types prefer writing, musical and artistic activities, while 
social types like activities that highlight their social skills such as teaching and counseling. 
Enterprising types prefer activities that capitalize on leadership and communication skills such as 
sales; conventional types like areas that require clerical and arithmetic abilities, such as 
bookkeeping. 

Interest as a construct has dominated the field of vocational psychology, but has also 
been shown to have a substantial impact on academic perfonnance. For example, interest in a 
particular academic domain relates to feelings of efficacy in that domain as well as productivity. 
Why may interests be particularly important to the graduate student domain? In a recent meta¬ 
analysis of the validity of the Graduate Record Examination, Kuncel et al. (2001) point to the 
superior predictive validity of the GRE subject tests over the GRE general test on academic 
performance and persistence. Tenned the interest hypothesis, the authors speculate that the 
subject tests may be measuring interest in a given subject matter, and the subsequent motivation 
to study and master that field. Thus, performances on the subject tests may be a product of 
acquired knowledge and ability as well as interest and motivation. In sum, while the GRE-V and 
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GRE-Q are primarily tests of ability, interest measures may be better predictors of motivationally 
determined aspects of graduate student performance such as effort and persistence. 

Measurement of interests. Among the ways to measure interests are self-reports, 
performance tests, and biographical data. 

Self-reports. Self-report tests are the most common way of measuring interests. Several 
scales have emerged from over 70 years of research, and currently reflect aspects of Holland’s 
(1959, 1973) theoretical framework. The Strong Interest Inventory (SII) (Harmon et ah, 1994) is 
the most often used interest assessment tool (Watkins, Campbell, & Nieberding , 1994). The 
scale is the culmination of continual updates and revisions of previous forms and is consequently 
extremely well researched and considered to be the most scientifically grounded interest 
assessment tool (Prince & Heiser, 2000). Previous versions of the SII include the Strong 
Vocational Interest Inventory (SVII) (Campbell & Hansen, 1981) and the Strong-Campbell 
Interest Inventory (SCII; Campbell & Hansen, 1981). The SII measures respondents’ work and 
personal interests and compares them to other people employed in a wide range of occupations. 
The scale is organized along four levels of scales. First, general occupational themes reflect 
Holland’s (1959, 1973) six types. Basic interest scales measure specific interests related to each 
occupational theme. For instance, agriculture reflects a basic interest item within the realistic 
occupational theme. Occupational scales reflect specific occupations such as physicist or fiction 
writer. Finally, the personal style scales include a work style scale, a learning environment scale, 
a leadership style scale, and a risk-taking/adventure scale. 

The Campbell Interest and Skill Survey (CISS; Campbell, Hyne, & Nilsen, 1992) also 
emerged from the Holland tradition. The CISS matches self-reported interests and self-assessed 
skills to occupations. The incorporation of skills into interest measures was highly innovative in 
that it integrated self-confidence into the assessment. Respondents rate their level of interest and 
level of skill on each item using a free response format, six-point scale. Interest and ability scores 
are assessed for several scales, which are similar to the SII, but focus on workplace activities and 
include additional scales such as academic focus and extraversion scales. The CISS was designed 
to be easier for practitioners to use and more sensitive to test taker diversity than previous 
interest measures. CISS outcomes are presented as the following patterns: pursue (high interest- 
high skill), develop (high interest-lower skill), explore (lower interest-high skill), and avoid (low 
interest-low skill). 
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Other interest inventories have been designed for individual use without the intervention 
of a vocational counselor. For instance, after the eventual refinement of his six types, Holland 
designed the Self-Directed Search (Holland, 1994) as a tool that would be easy for people to use 
to assist individuals in finding occupations that match their interests and skills. Respondents 
complete a series of scales (activities, competencies, occupations, and self-estimates) as well as 
an occupational daydreams section, which measures Holland’s six types. Based on responses to 
the scales, the individual can calculate a three-letter summary code that corresponds to a series of 
occupations and educational areas that match the code. For example, a code of SAR would 
indicate that an individual scored highest on social, next highest on artistic, and third highest on 
realistic, and the jobs and educational areas would correspond accordingly. The SDS is second 
only to the SII in most commonly used interest assessment tool. 

Another self-report assessment is the Harrington-0’Shea Career Decision-Making 
System (CDM) (Harrington & O’Shea, 1993). The CDM is an assessment of abilities, work 
values, and interests. Using a similar typology as Holland’s, the CDM measures six career 
interest areas: crafts (R), social (S), science (I), business (E), the arts (A), and office operations 
(C). There are also several computerized assessments, including SIGI PLUS, published by 
Educational Testing Service, DISCOVER, and Careerhub. Vocational interest inventories can be 
used to assess academic interests as well. Most interest scales suggest educational options in 
addition to occupations. For instance, the academic comfort scale of the SCII (Campbell & 
Hansen, 1981) and the academic focus scale of the CISS assess interest, motivation, and self- 
confidence within academic endeavors. Also, existing scales contain subscales that measure 
academic interests. The Dimensions of Self-Concept Scale Form H (DOSC) (Michael, Smith, & 
Michael, 1989) assesses students’ attitudes within the academic environment and includes a 
subscale for academic interest and satisfaction. 

Researchers have also developed scales to measure specific aspects of the graduate 
environment. For instance, Kahn and Gelso (1997) developed the Research Training 
Environment Scale-Revised (RTES-R), a measure of attitudes toward research among students in 
counseling graduate programs. The scale’s validity rests on its correlations with research interest, 
research self-efficacy, personality, and interest in practitioner activities. Kahn and Miller (2000) 
developed the RTES-R-S, a shortened form of the RTES, which demonstrated adequate internal 
consistency and corresponded closely to total scores from the original RTES-R. 
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Performance tests. Performance tests designed to measure abilities in various domains— 
such as SAT subjects test scores—are often combined with interest ratings obtained with self- 
report measures for an overall presentation of a student’s interests (Prince & Heiser, 2000). 
While some tasks appear to be more perfonnance-based, such as sorting out and prioritizing 
cards as in SIGI PLUS and TalentSort 2000 (Farren, 1998), they are closer to self-report. 

Biographical data. Biographical data might serve as a proxy measure of interest in an 
area. For example, college major, the number of courses taken in a given area, and dissertation 
topic all reflect interests. 

Predictors of interest. Among the predictors of interest are such factors as group, 
environmental, and personality. 

Group factors. Gender has traditionally been a major detenninant of interests 
(Gottfredson, 1996). Originally, separate interest inventories existed for men and women, each 
developed to accommodate areas such as housewife and nurse for women and engineer and 
businessman for men (D.P. Campbell, 1971), reflecting common response patterns. At the time 
of the revision of the Strong Interest Inventory in 1994, analyses revealed few important 
differences between men and women. 

Some gender differences remain. Men score higher on the realistic scale on the CDM, for 
example, while women score higher on the social scale (McLean & Kaufman, 1995). Women 
prefer occupations in the social and artistic domains, while men prefer more hands-on 
occupations (Slyter, 2001). Some differences are subtler—in an analysis of male and female 
psychologists, women report interest in opera while men report interest in the symphony, two 
obviously related areas of interest (Harmon et ah, 1994). An investigation of the interests of 
Midshipmen (undergraduate students) at the U.S. Naval Academy found women to score 
significantly higher than men on the Academic Interest and Satisfaction (AIAS) subscale of the 
DOSC-Form H (Paik & Michael, 2000). Thus, men and women do not differ on self-report 
interest measures to the extent that separate measures are necessary. 

Researchers have found few differences in occupational interests across various ethnic 
groups (Fouad, Harmon, & Hansen, 1994; Holland, Fritsche, & Powell, 1994), and little is 
known about the stability of interests as a function of ethnicity (Harmon, 1999). However, some 
have found interesting within-group gender differences. In an analysis of interests of men and 
women who belong to different ethnic groups, Tomlinson and Evans-Hughes (1991), using the 
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Strong-Campbell Interest Inventory, found that men reported higher realistic interests (consistent 
with past research) than did women. However, while Hispanic and White women were more 
likely than men to report artistic interests, Black women were less likely than Black men to 
prefer artistic interests. Further, Black men scored higher on the academic comfort scale than did 
Black women, suggesting the men to be more comfortable and satisfied with their educational 
experience than were the women. Also, Black women reported a preference for working alone, 
while Black men showed a preference for working with people. Additional research on group 
differences is clearly needed. Tinsley (1994) has emphasized the need to develop reliable and 
valid assessments and utilize existing theoretical frameworks to understand the relationship 
between vocational assessment and ethnic identity. 

Environmental factors. Presumably, students who undertake graduate school have a 
considerable amount of interest in their chosen field. Such an interest was likely influenced by a 
variety of environmental factors. Specifically, interests often develop through interactions and 
identification with a mentor or role model within a particular area (Savickas, 1999). Decisions to 
undertake graduate school may have been influenced through interactions with undergraduate 
advisors and revered faculty members. Also, numerous studies demonstrate that prior success in 
a domain increases interest in that domain (Locke, 1965; Campbell & Hackett, 1986). Therefore, 
interest among graduate students was likely fueled by undergraduate success in their chosen area. 

Personality factors. Interests appear to map onto the same fundamental constructs 
believed to underlie personality. An investigation of the relationship between the interests as 
measured by the Self-Directed Search and the Big 5 personality factors revealed substantial 
correlations. Notably, extraversion correlated with enterprising and social interests, openness 
correlated with artistic and investigative interests, and conscientious correlated with conventional 
interests (Costa, McCrae, & Holland, 1984; Gottfredson, Jones, & Holland, 1993; McCrae & 
Costa, 1997). Considering creativity and interests, artistic interests showed the highest 
association with creativity, followed by investigative, social, and enterprising. As one might 
expect, realistic and conventional interests were least related to creativity (Helson, 1996; 

Holland, 1986). Holland (1999) argues that personality and interests overlap and differ only in 
the criteria that underlie them. Ackerman and Heggestad (1997) have proposed a system of trait 
complexes or clusters that combine interests, personality, and ability variables. 
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Other research has revealed the relationship between self-efficacy and interests. Lent, 
Brown, and Hackett (1994) propose that people develop interests within domains in which they 
feel efficacious, and thus, anticipate positive outcomes. Several studies lend support to this idea. 
For instance, vocational interest was one of the strongest predictors of academic self-efficacy for 
engineering/science undergraduates (Hackett et al., 1992), and research self-efficacy is positively 
correlated with interest in the scientist domain (Geisler, 1996). Among graduate students in 
counseling psychology, research self-efficacy was positively correlated with responses on the 
research training environment scale, a measure that can be used to assess interests in research 
(Phillips & Russell, 1994). In a related vein, students who believe intelligence to be a fixed entity 
(i.e., not malleable; for more on this, see the Motivation section) report a decline in interest in the 
relevant academic domain in response to any negative feedback (Dweck & Sorich, 1999). 
Presumably, assuming that intelligence is unalterable decreases self-efficacy, and consequently, 
interest in the relevant academic domain. 

Self-efficacy scales are commonly used in conjunction with interest measures in 
vocational counseling settings. For instance, the Skills Confidence Inventory (SCI) (Betz , 
Borgen, & Hannon, 1996) measures self-efficacy with within each of the six Holland types. 
Further, the General Confidence Themes (GCT) consists of ten activities relevant to the Holland 
type. Respondents rate the extent to which they feel confident in their ability to complete the 
task. This provides interest/confidence patterns along the six personality areas and constitutes an 
important part of the overall strong profile. 

Interests as predictor. Numerous studies have demonstrated the link between interests 
and important academic outcomes. For instance, among counseling doctoral students, interest in 
research predicted research productivity and science-related career goals (Kahn & Scott, 1997). 
Swanson and Hansen (1985) found the Academic Comfort Scale of the Strong Campbell Interest 
Inventory to correlate with GPA and attained educational level among undergraduates. Academic 
Comfort scores were positively related to educational goals and graduate school plans, and the 
SCII was substantially more predictive of college majors for students with high Academic 
Comfort scores than it is for students with low AC scores. In an analysis of counseling graduate 
students, Phillips and Russell (1994) found a positive correlation between perceptions of the 
research training environment (PRTE) and research productivity (RP) only among a subsample 
of advanced graduate students. Finally, in an analysis of Ph.D. level counseling psychologists, 
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Krebs, Smither, and Hurley (1991) investigated potential influences of interests on research 
productivity (using Holland's Vocational Preference Inventory, 1985, and the Research Training 
Environment Scale (Royalty, Gelso, Mallinckrodt, & Garrett, 1986). Investigative interests were 
positively related to research productivity, whereas social interests were negatively related to 
research productivity. 

Other research has explored the link between interests and intelligence. Ackerman and 
Heggestad (1997) identified three of Holland’s interest domains that relate to intelligence: realistic 
interests, investigative interests, and artistic interests. Their review found realistic and investigative 
interests to be most closely related to fluid intelligence and abilities such as mechanical 
knowledge. Artistic and investigative interests are correlated with crystallized intelligence and 
verbal ability. Kaufman and McLean (1998) found that the investigative and realistic themes on 
the SII were significantly related to IQ, and the basic interest scales of writing, nature, teaching, 
mathematics, and art were also significantly related to IQ. McLean and Kaufman (1995), however, 
showed no relationship between IQ and interests as measured by the CDM. 

Social attitudes and values. For this review, social attitudes and values concern the most 
important and general values, beliefs, and attitudes that might reasonably be expected to affect 
success in higher education. Social attitudes and values may be distinguished from personality in 
several ways. One is that they include a valuation and valence component—whereas a 
personality statement is that “I enjoy being by myself,” a values statement might be “I believe it 
is a good thing to spend time alone.” They also are thought to be more modifiable than 
personality traits. Whereas an outgoing person tends to stay that way throughout his or her 
lifetime, we can and do change what we believe is important (e.g., happiness vs. success; 
material wealth vs. inner peace), as a result of our circumstances and interactions with others. 
Indeed studies of social attitudes and values are quite often studies of differences between 
cultures rather than individuals, under the presumption that culture which changes over time, and 
across borders, is manifested in social attitudes and values. Social attitudes and values are also 
more strongly associated with ideology than personality is (Saucier, 2003). 

There have been several broad approaches to studying social attitudes and values. Saucier 
(2000) summarized what might be considered the psychometric approach, focusing on an 
individual’s social attitudes and values. He concluded that much of social attitude research has 
been dominated by a few constructs—conservatism, authoritarianism, dogmatism, and 
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religiousness. Further, he argued that a strong general factor even ran through these, suggesting 
there really may be only one major social attitude factor. Using a lexical approach (similar to that 
used in studying personality), he was able to identify four social attitude factors, which might be 
labeled religious orthodoxy, unmitigated self-interest, protection of civil status quo, and 
unorthodox spirituality (Saucier, 2003). 

Focusing specifically on cultural rather than individual differences (typically differences 
between countries) is an alternative approach to studying social attitudes and values. Considering 
cultural differences, Schwartz (1992) proposed ten value dimensions: power, achievement, 
hedonism, stimulation, self-direction, universalism, benevolence, tradition, conformity, and 
security. The framework was later supported with a large-scale, cross-cultural study (Schwartz & 
Boehnke, 2003). Another approach was taken by Hofstede (2001), who administered a large 
values survey to workers (primarily IBM) in more than 50 countries. He identified five major 
dimensions: power distance (flat vs. hierarchical organization), uncertainty avoidance 
(conformity to rules), individualism, masculinity (competitive vs. relationship orientation), and 
long-term vs. short-term orientation. There are some obvious overlaps among the factors in these 
various schemes, but thus far no single study has attempted to link them. 

Measurement of social attitudes and values. The three major instruments—Saucier’s 
(2000), Schwartz’s (1992), and Hofstede’s (2001)—each present items to which the respondent 
indicates degree of endorsement on a Likert scale. Saucier’s instrument is based on dictionary 
definitions of “ism” words, such as patriotism, creationism, pragmatism, and the like (dictionary 
definitions are used because many of the ism words are not kn own to examinees). For each, 
examinees indicate their endorsement on a five-point scale from strongly disagree to strongly 
agree. Hofstede’s instrument is based interviews of IBM workers and existing surveys. It 
consisted of 160 to 183 items (depending on the form and language) asking a variety of questions 
about the workplace, such as how important (on a 1-5 scale) it was to “live in an area desirable 
to you and your family” and “have an opportunity for advancement to higher level jobs;” how 
satisfied “are you at present with ‘the challenge of the work you do,’ and ‘your fringe benefits,”’ 
and so on. Schwartz’s instrument, the Schwarz Value Survey (SVS), consists of 57 items, each 
requiring a 9-point Likert scale response. Items are words followed by brief explanations (e.g., 
EQUALITY [equal opportunity for all]), and the Likert scale ranges are “opposed to my 
principles,” “not important,” and “of supreme importance.” 
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Predictors of social attitudes and values . Saucier developed his scales to be orthogonal 
to personality. It is likely that there is some overlap between personality and the scales of 
Schwartz and Hofstede, but little research has been done to confirm this. There are major 
differences between countries with respect to social attitudes and beliefs, but summarizing these 
would go beyond the scope of this review. 

Social attitudes and values as preditors. There is considerable research linking social 
attitudes and values to all kinds of social and economic indicators of the kind that are measured 
at the national level, such as literacy, GNP, and life expectancy. Little if any research, as known, 
has attempted to link the kinds of social attitudes and values reviewed in this section to specific 
educational outcomes varying across individuals within a society. 

Prospects for applied use in admissions decisions. Attitudinal factors represent a rich 
source of student information that could be viable selection criteria for admissions committees. 
There are several problems with its use. First, most of the measures are self-report instruments. 
These would be subject to the problems of coachability and fakability that would plague 
personality measures used in admissions decisions. There were a few examples listed here that 
did not rely on self-reports. The CAVE method of analyzing attributional style might be applied 
to students’ writings, for example, as a way of determining whether students possessed adaptive 
attribution patterns going into graduate school. 

However, in addition to faking, there may be a couple of other considerations that would 
preclude acceptance of attitudinal measures in admissions. First, attitude may be rather 
malleable, perhaps more so than personality, and the quasi-cognitive factors. Thus, it might be 
appropriate to place more emphasis on attitude remediation and intervention strategies than on 
using attitude as a selection variable. Second, although attitude is certainly important, there may 
be less variance in attitude than in other noncognitive variables. It is likely that considerable self¬ 
selection will be involved—the applicant pool will likely consist of students who are closely 
identified with, interested in, and motivated to achieve within the domain. Attitudinal factors 
may not in fact be as valid as personality factors for graduate school admissions. It may be that 
attitudinal factors will have maximum value by serving the remediation needs of current students 
and also by informing applicants so that they may make advantageous application decisions. 

Summary. Attitudinal factors relate to an array of important achievement outcomes and 
often stem from environmental and group factors. While these factors are discussed in this report 
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individually, ultimately they all share substantial motivational underpinnings, which link to 
performance outcomes. For instance, one who is highly identified with a domain will likely be 
motivated to perform well by the personal satisfaction derived from success in that domain. 
Similarly, high self-efficacy may motivate a person to persist at a task because the person 
believes there is a high likelihood that the effort will lead to positive consequences. Work on 
theories of intelligence highlights the relatedness of these constructs and their relevance to 
performance outcomes (Dweck & Sorich, 1999). Specifically, theories about intelligence relate 
to beliefs about one’s self-efficacy and choices of motivational goals, which in turn influence 
effort and perfonnance. 

Others have proposed that self-esteem (the value attached to one’s self-concept), self- 
efficacy, and locus of control, combined with emotional stability (a personality trait), comprise a 
higher order construct called core self-evaluations (Judge, Locke, & Durham, 1997). Core self- 
evaluations were found to be an excellent dispositional predictor of job satisfaction and job 
performance (Judge & Bono, 2001). Investigations on the link between such factors and higher 
education outcomes would be useful. 

Attitudinal factors undoubtedly influence performance on the GRE and related graduate 
school success determinants such as effort and persistence. They likely also affect the 
relationship between the GRE and outcomes, that is, the test’s predictive validity. For instance, 
one could have an adaptive attitude regarding graduate school, but a maladaptive one regarding 
the GRE (e.g., low motivation or low self-efficacy), which would affect the validity of the GRE. 
Or, as another example, students who are not identified with the relevant academic domain or 
who do not have a close match between their interests and their area of study may not succeed in 
graduate school despite high GRE scores. Even if there are problems with the measurement of 
attitudinal factors per se, it is nevertheless useful to understand these relationships. 

This review suggests that graduate school persistence and perfonnance is enhanced by 
awareness on the part of students, faculty, and admissions decision makers of these attitudinal 
factors. Future work is needed to detennine the viability of assessing attitudinal factors for 
purposes of graduate student self-assessment for interventions, and perhaps graduate school 
admissions purposes. 
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Environmental Influences on Graduate School Outcomes 

A variety of environmental factors play important roles in graduate student outcomes 
both directly and indirectly through their effect on personality and self-views. The existing 
literature describes a wide array of factors ranging from prior academic experiences to social 
integration and financial support. These factors include mentor support, social support, 
encounters with academic stereotypes, institutional integration, nonacademic 
demands/influences, financial support, and prior accomplishments/academic experiences. 

Mentor support. Students’ interactions with faculty mentors can be a critical component 
of the graduate school experience. The quality of student/faculty interactions may affect student 
outcomes directly through learning opportunities and experience, and also indirectly through the 
student’s orientation and integration into the academic environment. 

Pascarella (1980) proposed a longitudinal model of academic persistence and institutional 
integration that emphasized the importance of informal contact with faculty. His model proposes 
that contact that transcends the scope of normal classroom activity fosters intellectual or 
interesting social dialogue, which ultimately motivates students. Conversely, the absence of such 
informal contact is believed to be the single most important predictor of attrition even after 
taking into account student background and personal and academic performance. Aspects of 
Pascarella’s model have received empirical support (see Terenzini, Pascarella, & Blinding, 1999, 
for a full review). Moreover, in Lovitt’s (2001) influential work on graduate student attrition, she 
found that both the nature and the quality of the advisor-advisee relationship to be a primary 
factor in the graduate experience. 

Measurement of mentor support. The Ideal Mentor Scale (Rose, 2000) was designed to 
assess graduate students’ definitions of the ideal mentor. Three underlying factors emerged 
through factor analysis: integrity, guidance, and relationship. Among Ph.D. students, Rose 
(2000) finds that nearly all respondents defined an ideal mentor as an experienced person who 
exhibits intellectual curiosity, reliability, research ethics, and good communication skills. They 
are available to the student, provide challenge and constructive criticism, and convey a belief in 
the student’s capabilities. Bennett (2001) explored factors that constitute effective mentoring by 
comparing the outcomes and experiences of students who reported having a mentor with those 
who reported having an academic advisor. Presumably, a mentor relationship constituted a closer 
relationship than one characterized as an advisor. 
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Mentor support as a predictor. Lovitt (2001) found positive graduate experiences to be 
strongly related to advisors’ productivity, academic and social engagement with the department, 
accessibility, and interest in students’ academic and professional development. Moreover, a wide 
sampling of doctoral students found perceived mentor support to predict students’ self-reported 
level of commitment to their chosen field (Uelkue-Steiner et ah, 2000). Doctoral students in 
education programs who reported more positive and cooperative relationships with their advisors 
and committee members were more advanced in their dissertation writing (Faghihi, 1998). 
Similarly, among science and engineering graduate students, expectations about the quality of 
student/faculty interactions was a significant predictor of feelings of academic self-efficacy and 
expectations for career-related outcome expectations (Santiago & Einarson, 1998). In an analysis 
of institutional factors associated with persistence in doctoral programs, Strieker (1993) found 
that faculty accessibility, defined by the student to faculty ratio and department size, correlated 
with average time to completion of the doctorate degree among psychology students, but not 
among chemistry and English students. Thus, mentor support appears to relate to important 
graduate outcomes, but the type and magnitude of the relationship may vary across departments. 

Mentor support may be particularly critical among students of color and women. Among 
Black undergraduates at PWIs, the number of Black faculty at the university is positively 
correlated with graduation rates (Steele, 1991). In an analysis of Black and Hispanic doctoral 
students, students who persisted to complete their degree were more likely to report having 
supportive major advisers (Clewell, 1987). Further, Black doctoral students who reported having 
had academic mentors reported greater mentor effectiveness and overall satisfaction with their 
doctoral programs than did students who reported having academic advisors, implying a more 
distant and formal relationship. Also, female mentors received higher effectiveness ratings than 
did male mentors from students (Bennett, 2001). A report from the American Council of 
Education identified faculty support, an atmosphere of expected success, faculty mentoring, and 
a critical mass of minority students and faculty as crucial factors in creating supportive 
environments to increase the participation and success of students of color in doctoral programs 
(Wagner, 1992). 

In support of Pascarella’s model among female students, the most significant positive 
effect on college persistence came from mentoring experiences in the fonn of nonclassroom 
interactions with faculty (Nora et ah, 1996). Among 81 female graduate students over age 30 
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(nontraditional students), the quality of the student/faculty interactions was among the most 
important predictors of persistence (Hagedorn, 1999). In an analysis of graduate students of 
color, quality of teacher-student interactions and strong faculty support were among predictors of 
success for graduate students of color (Clewell & Ficklen, 1986). 

Bandura’s (1977) social learning theory can be extended to research on the 
student/mentor relationship. A review of the research has shown that students’ educational 
outcomes improve when the faculty is comprised of others who share their group membership, 
and this effect may be due in part to students’ vicarious learning processes. Specifically, students 
may learn optimal strategies for achievement by observing the behaviors of successful similar 
others. Clearly, a quality mentor relationship provides the student with guidance as well as 
support. 

Social support. Access to supportive others has long been recognized as an important 
factor in health, emotional well-being, and achievement outcomes. Like mentor support, support 
from one’s social group is generally viewed as an important factor in all aspects of the graduate 
student experience. Some graduate schools have attempted to form student cohorts in an effort to 
promote the retention of graduate students (Dorn & Papalewis, 1997). 

Measurement of social support. Several scales are used to assess perceptions of social 
support. For instance, the Multidimensional Scale of Perceived Social Support (MSPSS) is 
commonly used to assess perceptions of social support from family, friends, and significant 
others (Zimet, Dahlem, Zimet, & Farley, 1988). It has proven to be a valid and reliable measure 
of social support among Black students (Canty-Mitchell & Zimet, 2000). Other scales have been 
developed specifically for use with college students. For instance, the Young Adult Social 
Support Inventory (Grochowski & McCubbin, 1984) assesses the social support systems 
available to college freshmen. Information on perceptions of supportive others and available 
campus resources is often collected via informal student surveys. 

Social support as a predictor. Data from a survey of doctoral students from eight 
universities suggests that group cohesiveness and persistence to attaining the degree are 
significantly correlated (Dorn & Papalewis, 1997). 

Among female graduate students over age 30, quality of student/student interactions was a 
primary predictor of persistence in addition to quality of student/faculty interactions (Hagedorn, 
1999). In an analysis of graduate students in the 1970’s, Black and Hispanic females consistently 
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reported lower academic self-concept than males, and the author attributed this to a graduate 
school racial-climate with low trust and little interaction among groups (Hurtado, 1994). 

The report from the American Council of Education also recognized social support as one 
of the factors crucial to increasing the participation and success of Black, American Indian, 
Hispanic, and Asian American students in doctoral programs (Wagner, 1992). Similarly, in an 
analysis of noncognitive influences of academic success, access to a strong supportive system 
was a primary predictor of persistence among students of color (Tracey & Sedlacek, 1984). 

Encounters with academic stereotypes and prejudice. Regardless of one’s ethnicity, 
belonging to a group that is a minority on a college campus is related to feeling racially 
discriminated against. Among graduate students attending PWIs, Black and Hispanic doctoral 
students perceived substantially more racial discrimination than did White students (Nettles, 
1990). Tan (1994) examined factors related to Asian American students’ participation in higher 
education, their academic performance, and their experiences as college students when compared 
with students from other ethnic minority groups. While Asian American students had higher 
rates of academic participation and higher GPA’s than did Black students, both groups reported 
facing many incidents of racism and prejudice, primarily from fellow students. Further, both 
groups expressed having had a lack of success in coping with these incidents. 

Particularly problematic for women in mathematics and science fields and students of 
color is the experience of encountering negative stereotypes about their groups within the 
graduate academic domain. Stereotypes about intelligence are widely known, even among people 
who are targets of the stereotypes and who do not endorse them (Devine, 1989), and researchers 
have recently focused on the impact of these stereotypes on targets within the educational 
environment. A program of research has developed around the concept of stereotype threat, 
defined as a predicament in which a person fears verifying a negative stereotype about one’s 
group (Steele, 1997). The threat arises in situations in which a stereotype about one's group is 
salient, and the group member becomes apprehensive about inadvertently confirming the 
stereotype. For instance, a female mathematics student might fear being evaluated according to 
the stereotype that women are not good at mathematics prior to an important mathematics exam. 

Any person who is identified with the academic environment and who belongs to a group 
to which a stereotype about intelligence applies will be susceptible to stereotype threat. For 
instance, common perceptions that Asian American students are exceedingly capable at 
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quantitative tasks may introduce excessive pressure for these students (Tan, 1994). In contrast, 
Black and Hispanic students are often subjected to stereotypes that question their intellectual 
ability (Niemann, Jennings, Rozelle, & Baxter, 1994). As will be discussed, the experience of 
stereotype threat has been shown to suppress performance within important academic domains. 
Thus, stereotype threat has garnered attention in social psychological and educational research. 

Measurement of encounters with academic stereotypes. Steele and Aronson (1995) 
administered an eight-item questionnaire designed to measure students’ self-reported experiences 
with stereotype threat. Students reported their agreement to items such as “Some people feel I 
have less verbal ability because of my race,” and “The experimenter expected me to do poorly 
because of my race.” Also, often researchers simply ask students of color about their experiences 
with prejudice and discrimination. For instance, Brown (1997) asked students’ level of 
agreement with statements including “Prejudice is still a major problem in the United States,” 
“People in my group often face prejudice and discrimination,” and “I have been treated 
negatively because of my ethnicity/race.” 

Academic stereotypes and prejudice as a predictors. Researchers have demonstrated that 
stereotype threat often leads a target to inadvertently confirm the stereotype the individual 
wished to avoid. For example, in one study, Black students who were told that they would be 
taking a test that measured their intellectual ability, thus priming the negative ethnic stereotype 
of low intellect, performed worse on a test in comparison with Black students who were told that 
the test was merely an analysis of the problem solving process (Steele & Aronson, 1995). 
Apparently, the mere introduction of a negative stereotype can hinder performance. As expected, 
Black students reported significantly more experience with stereotype threat than White students, 
as measured by a stereotype threat questionnaire. 

Researchers have demonstrated the consequences of stereotype threat with a variety of 
populations. College-age women performed worse than men on a difficult mathematics test when 
told that the test revealed gender differences (Spencer, Steele, & Quinn, 1999). However, women 
who received no such information performed equal to men on the same test. Further, Asian 
American undergraduate students who encountered stereotypes about their superior intellectual 
ability were less able to concentrate and, as a consequence, perfonned poorly compared to 
students who were not reminded of the stereotype (Cheryan & Bodenhausen, 2000). Although 
people commonly hold positive stereotypes about Asian students' mathematical skills, making 
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these stereotypes salient prior to performance can create the potential for "choking" under the 
pressure of high expectations. 

Levy (1996) demonstrated that stereotype threat is not limited to stereotypes about gender 
and ethnicity. Elderly participants performed poorly on a memory and motor skills task only 
when reminded of negative stereotypes about the elderly such as dementia. Others have extended 
this work to social class (Croizet & Claire, 1998). The correlation between socioeconomic status 
and intellectual ability is often demonstrated and widely known (Neisser et ah, 1996). Therefore, 
students from low socioeconomic backgrounds performed worse than other students on a test 
when it was described as diagnostic of intellectual ability. Collectively, these studies suggest that 
simply making a negative self-relevant stereotype salient within a setting that is relevant to that 
stereotype can hinder perfonnance. 

Although the causes of stereotype threat remain uncertain, research has suggested that 
perceptions of important evaluative audiences (i.e., mentors, instructors and test administrators) 
may be key (Walters et ah, 2001). So expectations of being stereotyped when in an environment 
where a negative group stereotype is salient can have deleterious consequences. Prior research 
describes these perceptions as meta-stereotypes—one’s beliefs about the stereotypes that others 
hold about one’s group (Vorauer, Main, & O’Connell, 1998). This research has shown that meta¬ 
stereotypes can lead to a threat to self-concept (Vorauer, Main, & O’Connell, 1998). Other 
research has supported the notion that stereotype threat stems from social aspects of the testing 
environment. Inzlicht and Ben-Zeev (2000) found that test perfonnance could vary with the 
social composition of the testing environment. Specifically, when taking a mathematics test with 
other students, the more male students that were present, the worse the female students 
performed. 

Ultimately, chronic exposure to stereotype threat may lead a threatened student to 
disidentify from the academic domain (Steele, 1997). Specifically, students will come to detach 
their identity from the domain in which they feel devalued, and consequently be unable to 
perform adequately (Crocker & Major, 1989). While graduate students of color and female 
graduate students have most likely found ways to persist despite these circumstances, these 
factors may still contribute to higher attrition rates than White and male graduate students 
(National Center for Educational Statistics, 1995; Zwick, 1991). 
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However, researchers caution against misinterpreting stereotype threat findings and 
overgeneralizing the effect to actual testing and educational environments (Sackett et ah, 2001). 
For instance, in one analysis within an operational testing setting, Strieker and Ward (1998) 
failed to find stereotype threat. Others question the conclusions drawn about stereotype threat, 
because research has not yet considered the role of voluntary and involuntary minority status in 
performance outcomes (Ogbu, 2001; Ogbu & Simons, 1998). Future research will need to 
investigate the generalizability of stereotype threat research. 

Women and students of color must address a greater variety of circumstances beyond 
academic performance. In an analysis of noncognitive predictors of academic success, the ability 
to deal with racism significantly predicted success among students of color (Tracey & Sedlacek, 
1984). Consistent findings that women at both graduate and undergraduate levels report that 
feelings of psychological alienation or depression played a critical role in their decisions to leave 
science/mathematics/engineering disciplines, and that despite good academic performances, they 
experience diminished self-esteem, self-confidence, and career ambitions (Hewitt & Seymour, 
1997). 

Institutional integration (academic and social). Perceptions of mentor and social 
support in addition to experiences with stereotypes influence the extent to which students feel 
socially and academically integrated into their university or graduate program. Academic and 
social integration into the institution may depend on a variety of factors such as the student’s 
group membership, the majority ethnic and gender composition of the institution or program, in 
addition to general influences of social and mentor support. 

Student integration is the key component of Tinto’s (1975) well-known model of student 
retention. Tinto views retention and attrition in higher education as longitudinal processes that 
are a function of students’ integration, or lack of, into the academic and social climates of the 
institution. He proposed that each student enters the institution with a unique set of background 
characteristics (beliefs, personal and family issues, academic characteristics and commitment, 
and dispositions) with respect to college attendance and goals. The student’s level of social and 
academic integration is continually being modified with everyday exchanges with members of 
the academic and social systems of the college or university. Satisfying academic and social 
encounters lead to increased integration, thereby resulting in higher rates of retention. 
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Further, Lovitt (2001) highlighted the role of academic and socioemotional integration in 
graduate school persistence. Her work indicated that these factors serve as vehicles for access to 
vital information about the formal and informal expectations for degree attainment. This 
knowledge constitutes cognitive maps, which guide students through the necessary requirements 
of each phase of their programs. Survey and interview data revealed that nondegree completers 
experienced fewer opportunities for integration into the department community, and 
consequently, received little guidance, infonnation, and support necessary to complete their 
programs. 

Measurement of institutional integration. Institutional academic and social integration 
are often assessed through structured interviews with graduate students. However, in an effort to 
test Tinto’s model and quantify these experiences, Pascarella and Terenzini (1983) created the 
Institutional Integration Scale (IIS) to measure students’ academic and social integration into 
college. Academic integration is measured by the Academic Development Scale (alpha = .72) in 
conjunction with GPA, credits earned, and hours spent engaged in academic extracurricular 
activities (such as band, theatre, and professional organizations). Social integration is measured 
by the Peer Group Relations Scale (alpha = .84), the Infonnal Faculty Relations Scale (alpha = 
.83), as well as measures of residency, campus employment, hours spent engaged in social 
activities, and hours spent engaged in intercollegiate athletics. 

Predictors of institutional integration. Tinto (1975) proposed that successful academic 
and social encounters lead to increased integration into the university environment, thereby 
resulting in higher rates of retention. Further, students’ motivational orientations can influence 
these patterns of integration and attrition. For instance, among students who were motivated to 
attend college for practical, academic, or knowledge-seeking reasons, academic integration 
positively and directly influenced persistence (Stage, 1989). 

An important factor in institutional integration is the predominant ethnicity at a 
university. For instance, Black students attending HBCUs present reported greater academic 
integration than did Black students attending other institutions (Robinson, 1990). They also had 
higher GPA’s and graduation rates. In a longitudinal study of Black students, attendance at an 
HBCU led to significant increases in standardized measures of reading comprehension as well as 
self-reported measures of increased understanding of the arts, humanities, and science (Flowers 
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& Pascarella, 1999). Thus, integration and consequent academic success may depend in part on 
students’ shared group identity with other students at the institution. 

Although comprehensive programs and policies geared toward minority student retention 
have generally been successful (Clewell & Ficklen, 1986), some efforts by institutions to address 
the needs of underrepresented groups within graduate programs may have an adverse effect on 
integration. For instance, among women entering science, mathematics, and engineering majors, 
some reported that preferential hiring policies, scholarships which promote women’s 
participation in science, and special programs to support women through SME majors added to 
the difficulties which they already faced in establishing male respect of their abilities (Hewitt & 
Seymour, 1997). Such women avoided any kind of special treatment, did not use women 
advisors or programs, and did not join women’s professional societies. They preferred to cope 
with academic difficulties by distancing themselves from all sources of official help based on 
gender. Similarly, undergraduate students of color often avoid opportunities that might make 
their ethnic identity salient such as gifted programs designated for minority students (Fordham & 
Ogbu, 1986). These researchers suggest that these problems can be addressed effectively by 
reviewing student performance (both minority and nonminority) to detennine where systemic 
underperformance exists, and then to take steps to address those areas. 

However, it is important to note that several groups of students of color often persist and 
achieve despite underrepresentation of other group members at their institution and other sources 
of support. Again, future tests of models of institutional integration will need to consider the 
wide range of experiences that influence the needs of individual groups (Ogbu, 2001; Ogbu & 
Simons, 1998; Ogbu & Stem, 2001). 

Institutional integration as a predictor. Lovitt’s (2001) survey and interview data 
revealed that nondegree completers experienced fewer opportunities for integration into the 
department community, and consequently, received little guidance, information, and support 
necessary to complete their programs. In an investigation of students at 23 colleges and 
universities, a positive significant relationship was found between performance outcomes 
(including standardized test scores and self-reported performance gains) and peer interactions 
(including course-related and non-course-related activities occurring inside and outside 
classroom settings; Whitt, Edison, Pascarella, Nora, & Terenzini, 1999). 


84 



In support of Tinto’s model of student retention, Wood (1995) found that social support 
among freshman influenced integration into the university and subsequent persistence through 
the second year. Further, Clewell (1987) found that participation in professional activities in 
graduate school differentiated persisters and nonpersisters among Black and Hispanic doctoral 
students. 

Nonacademic demands/influences. Nonacademic demands and influences among 
graduate students describe a wide range of effects that may be beneficial and detrimental to 
academic performance outcomes. 

Measurement of nonacademic demands/influences. Infonnation about students’ 
nonacademic commitments can be gathered by demographic questionnaires and structured 
interviews. The Multiple Role Conflict Scale (MRCS) measures psychological strain of handling 
multiple roles (Middlesworth, 1999). Although the scale was designed to assess the mental 
health correlates of women engaged in multiple roles, the scale could be modified for male and 
female graduate student populations. 

Nonacademic demands/influences as a predictor. Multiple role conflict negatively 
related to academic self-efficacy among graduate students in education (Feldmann & Martinez- 
Pons, 1995). For minority students, family responsibilities and working off-campus were the 
strongest predictors of attrition (Nora et ah, 1996). However, extracurricular activities can also 
have a beneficial role. Tracey and Sedlacek (1984) found community involvement to be a 
significant predictor of persistence among Black students. 

Financial support. Graduate students may receive various fonns of financial support 
from their institutions including, but not limited to, fellowships, teaching assistantships, and 
research assistantships. 

Financial support is important. It promotes staying in doctoral programs (Attiyeh, 1999; 
Jacobson, 1992; Lovitt 2001), and leads to better dissertations (Faghihi, 1998). Raising tuition 
increases attrition (St. John & Andrieu, 1995). 

A question is, what is the best kind of financial support—fellowships or assistantships? A 
survey by economist Ronald G. Ehrenberg at Cornell University found that doctoral students are 
more likely to complete their degrees, and in less time, if they receive fellowships rather than 
research or teaching assistantships (Jacobson, 1992). Lovitt (2001) found evidence to support the 
opposite—students with fellowships had higher rates of attrition than did students with research 
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and teaching assistantships. Lovitt posited that fellowships inhibit opportunities for academic and 
social integration because students have fewer on-campus obligations. In support of Lovitt’s 
thesis was a study of fonner Ph.D. students from the University of Buffalo who reported that 
financial support with a service commitment led to a fuller graduate experience, as well as to 
shorter time to degree (Border & Barba, 1998). 

Which is right? There is no obvious reason for the discrepancy; perhaps the best that can 
be done is simply to list the differences between the studies. Ehrenberg’s sample included 
students in 4 departments across 25 years at Cornell, while Lovitt studied a single cohort of 
students from nine departments from one urban and one rural university. Lovitt’s departments 
varied in their financial support practices, whereas Ehrenberg’s departments were similar. 
Ehrenberg statistically controlled for gender, aptitude, citizenship, prior education, and starting 
salaries within each discipline, Lovitt did not. The question of whether unencumbered 
fellowships are more or less effective than service-committed assistantships is an important 
issue, but the differences between studies does not help here—not enough is known now to say 
which form of financial support is better. 

Another question is whether assistance is equally accessible to all students. The answer 
seems to be no: Women in science and engineering graduate programs (Dix, 1987), graduate 
students in urban schools, and humanities students (Lovitt, 2001) are all less likely to find 
financial aid. It seems safe to assume that fellowships and assistantships would increase the 
participation and success of minorities in doctoral programs (Wagner, 1992). So it may seem 
surprising to find that Black and Hispanic graduate students with the highest attrition rates were 
also more likely than White and Asian American students to have received fellowships (Lovitt, 
2001). Lovitt’s explanation is that fellowships may be useful in recruiting students of color, but 
they may also diminish opportunities to integrate into the environment. 

Prior accomplishments/academic experiences. The quality and type of students’ prior 
experiences have an obvious impact on graduate school experiences and outcomes. Research has 
considered such factors as undergraduate and high school coursework, achievement test scores, 
as well as a wide range of service and internship opportunities. 

Measurement of prior accomplishments/academic experience. Transcripts provide 
important information about the content and performances in prior coursework. Self-report 
measures often included in graduate applications (e.g., relevant coursework) can also yield prior 
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educational information. Baird (1979) created a prior accomplishments inventory for graduate 
school applicants that included literary and expressive, scientific and technical, artistic, and 
social service organizational activities (Baird and Knapp, 1981). Strieker (1983) created a 
documented accomplishments scale, designed to enable validation, that is organized into 
academic achievement, leadership, practical language, aesthetic expression, science, and 
mechanical domains. 

Prior accomplishments/academic experiences as a predictor. Several studies have 
demonstrated the relationship between prior accomplishments and academic experiences and 
later academic success. Baird and Knapp (1981) found that pre-graduate school accomplishments 
predicted a host of graduate school accomplishments, although not graduate school grades. 
Similarly, participation in extra-curricular activities positively correlated with retention and 
graduation among undergraduates (Pascarella & Terenzini, 1991). In a comparison of students 
who did and did not graduate from undergraduate business-related programs, graduates were 
more likely to have participated in pre-career programs such as Future Business Leaders of 
America (Robinson, 1990). 

Surprisingly, group differences have not been found on measures of accomplishment 
among GRE examinees (Strieker et al., 2001), with an exception of men reporting more 
mechanical accomplishments than women. High achievement as early as high school has been 
shown to predict persistence among Black and Hispanic doctoral students (Clewell, 1987). Still, 
inadequate preparation via coursework or prior experiences has been identified as a possible 
source of differences in graduate student success between men and women in the sciences (Dix, 
1987). Self-efficacy, which may partly reflect differences in preparation, may also have an 
influence (Mulkey, 1997; Pajares & Miller, 1994). 

Summary and Discussion 

Several GRE studies conducted in the past several years have asked faculty about the 
qualities important to success in graduate school. A consistent finding is that graduate faculty 
believe that noncognitive factors, such as motivation, creativity, personality, interests, and 
attitude, ought to be considered in graduate admissions. Many faculty believe that including such 
factors would increase both fairness and the validity of the current admissions process. At the 
same time there is a lack of knowledge about what the scientific literature says on what the 
noncognitive factors are, how important they are, how they could best be measured, and on 
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whether it might be practical to use them in admissions decisions. The purpose of this review 
was to address these questions. This report identified various noncognitive factors and reviewed 
definitions, measures, correlates, and the validity of those measures. 

We found it useful to divide noncognitive factors into predictor and criterion (i.e., 
outcome) groups, and to further divide noncognitive factors into several additional categories. 
The key predictor factors we evaluated were personality, attitudinal, and quasi-cognitive factors. 
To get a full picture, we also considered the role of various background factors—environmental 
factors, gender, and race. 

Based on this review, we believe that there are four areas of potential application of 
noncognitive factors warranting consideration by the GRE: admissions, guidance, policies and 
interventions, and outcome evaluation. 

Admissions 

There are several advantages to including noncognitive factors in graduate admissions. 
One is to broaden the range of qualities considered. There are numerous reasons why students 
fail to complete graduate programs successfully, and only some reasons are strictly cognitive—it 
makes sense to consider important noncognitive qualities such as interests, motivation, ability to 
work with others, and willingness to work hard, in evaluating a candidate’s potential for 
successful graduate study. Moreover, faculty have specifically stated in surveys and focus groups 
that they believe it is important to consider a wide range of such noncognitive factors in addition 
to cognitive ones in admissions decisions. Another advantage of including noncognitive factors 
is that gender and race differences on such factors tends to be much smaller than those observed 
for cognitive factors, and in many cases, differences are eliminated altogether. Adding such 
factors to those for which gender and race differences do exist will decrease overall score gaps 
and could lead to increased admissions of ethnic minority students and women in various 
academic programs. Such an expansion of variables considered in admissions therefore would be 
responsive both to validity and fairness goals. 

There are also some problems in including noncognitive factors in graduate admissions. 
The primary problem is that currently, noncognitive factors of the type faculty often mention are 
most typically measured with self-reports. Self-reports are coachable and fakable. There are 
numerous methods designed to minimize faking, but these have never been evaluated in the 
context of a large-volume tests used in making high-stakes decisions such as the GRE. The 
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coaching industry would quickly crack the code on any noncognitive measure and translate that 
into a successful strategy for defeating the measure if it were a self-assessment. 

Performance measures of noncognitive factors. There may be ways to get around this 
problem. One is through the development of a skills or perfonnance measure (sometimes 
referred to as an objective, or objective-analytic measure in the personality literature). Many 
emotional intelligence measures, at least the ones that are not self assessments, may be 
considered skills or performance measures, and may be relatively resistant to coaching and 
faking. Such measures have not been validated against operational criteria yet; however, it seems 
reasonable that they could prove valid in some disciplines, such as clinical psychology, social 
work, nursing, and other disciplines involving interpersonal skills. Other performance measures, 
even less well developed, are the Implicit Association Test (IAT) (Greenwald & Benaji, 1995) 
and the conditional reasoning task (James, 1998). It is not known whether any of these could be 
developed for the noncognitive factors of interest in graduate admissions, whether they would be 
coachable, or whether they would be valid. But the approaches seem promising and warranting 
of further research and further consideration. 

Standardized letters of recommendation and personal statements. Another approach 
is to build on the mechanisms already in place for transmitting noncognitive information to 
faculty. In the interviews, faculty said they obtained such infonnation from letters of 
recommendation and personal statements. A major problem with these two mechanisms is that 
no one kn ows how successful they are in perfonning the task of communicating noncognitive 
information about the student to the admissions committee. Faculty like them and use them, but 
few if any large-scale systematic studies have been conducted on their actual validity. There are 
several ways this can be done: 

1. Coding letters of recommendation with respect to various noncognitive dimensions 
(e.g., general personality factors, motivation factors) and conduct validity studies 
from the coded infonnation 

2. Creating standardized letters and have faculty complete such evaluations of students 
(a letter of recommendation) or have students complete such an evaluation (a personal 
statement). In the case of the latter, it might be useful to have a second party (such as 
a faculty advisor) review the student’s personal statement to avoid the self-assessment 
bias that is possible in all noncognitive self-assessments. 
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Guidance 

There are guidance systems, such as SIGI PLUS from ETS, that use noncognitive 
variables such as the ones discussed in this review. The most popular are interest inventories, and 
many guidance systems are designed to match a student’s or an applicant’s interests with the 
requirements of a job or field of study. No guidance system has attempted to match the full range 
of noncognitive variables—personality, attitudinal, and quasi-cognitive factors, along with 
background variables—with the requirements or characteristics of graduate schools, 
departments, or disciplines. However, it might be useful to think about the value of such a 
system, and how it could be implemented. Suppose one could measure a large number of 
noncognitive variables on graduate students in particular programs, departments, schools, and 
disciplines, in an effort to develop a profile of what such students are like. One could then 
measure the same variables on applicants, who then could be presented with feedback on their 
own profiles and a presentation of the profiles of various programs to which they are considering 
applying. This could be useful information in selecting degree programs. One could also conduct 
follow-up studies to evaluate whether student/department profile matches lead to better graduate 
school outcomes. 

Policies and Interventions 

This review suggests that institutional policies and interventions be designed to increase 
graduate school outcomes, particularly for certain groups, such as ethnic minorities, or women in 
mathematics, science, and engineering programs. This review specifically included discussion of 
race and gender mediation effects on graduate school outcomes for all the noncognitive factors. 
Further review of some of these findings may provide some suggestions on where additional 
support policies, and additional interventions might productively be developed. 

Outcomes 

In the interviews and focus groups reviewed here, faculty identified the importance of the 
various noncognitive factors not only for admissions purposes, but also as an outcome of 
graduate study. For example, faculty mentioned the importance of developing qualities such as 
persistence, collegiality, independence, creativity, enthusiasm, and good values as part of the 
graduate school experience. One of the purposes of the review was to define these qualities that 
faculty mentioned more precisely; to assemble the research literature on these qualities, and to 
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note their precedents, antecedents, and correlates. Having done so, it should now be possible to 
assemble a taxonomy of such factors, along with methods for adequately measuring them. Such a 
resulting battery of measures could serve as a guide for a comprehensive noncognitive outcomes 
assessment, which could be administered to graduate students perhaps at the beginning and end 
of graduate school. Comparing scores from these two time points could inform graduate faculty 
and graduate schools about the effect of their graduate programs on these less tangible 
noncognitive factors. Or profdes of students who leave could be compared with students who 
completeg the degree in studies of the causes of attrition. All these would be low-stakes 
applications, and therefore it would be possible to use the self-assessment instruments without 
the same fear of measurement bias that would threaten admissions applications. Institutions 
could conduct longitudinal studies of such factors for internal or promotional purposes to 
illustrate a range of outcomes for students of their programs, thus going beyond the knowledge 
and skills mastery outcomes normally touted. 

Summary 

The purpose of this review was to consider how the graduate education community could 
use noncognitive constructs and measures. It is important to start by considering what 
noncognitive factors faculty and those involved in graduate admissions value, both in admissions 
and as program outcomes. Fortunately, there have been a couple of recent studies of these factors 
that provide the beginnings of a good sense for what those qualities are, and a useful framework 
from which to interpret findings from those studies. The world of noncognitive variables is large, 
but it is manageable. Significant strides have been made in our understanding of the space of 
noncognitive variables, to the point where it is now productive to begin validating those 
variables with respect to graduate school outcomes, rather than continue exploring that space, per 
se. There are certainly varied categories of noncognitive variables—from group and 
environmental variables to personality. Considering these together, as attempted in this review, is 
important for theorizing about how noncognitive variables affect graduate school outcomes. 

Finally, there are specific opportunities in noncognitive assessment for graduate 
education that could be pursued immediately. For the most part, doing something like 
administering a personality inventory as a potential supplement to the GRE-V and Q is probably 
not a realistic possibility given the threat of coaching and fakeability. But there are a number of 
other ways in which noncognitive variables might be used—in admissions, guidance, policy, and 
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outcomes assessment. In admissions, one can imagine using noncognitive variables in the 
creation of a guide for writing or interpreting letters of recommendation or personal statements. 
In guidance, there is a possibility of using noncognitive information to help students find 
compatible graduate study programs. Policy recommendations and interventions could be 
developed based on relationships identified here between noncognitive factors and graduate 
school outcomes. Outcomes might be expanded, and institutional studies conducted, based on the 
inclusion of a wide range of noncognitive outcome variables in comparisons of beginning and 
finishing graduate students, or those who leave and those who stay. 
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Notes 


1 That is, adding extroversion and agreeableness scores, and subtracting conscientiousness scores 
would yield a score highly positively correlated with Snyder’s self-monitoring score. In other 
words, high self-monitors are those high on extroversion and agreeableness, and low on 
conscientiousness. The authoritarian personality is associated with being high on 
conscientiousness, and low on agreeableness and openness. 

' Eysenck (1991) turned this argument around by asserting that conscientiousness and 
agreeableness were (atheoretical) facets of psychoticism. 

This comprehensive list was assembled by Kamp & Hough (1988) in their classification of 
personality dimensions for the US Army’s Project A (Campbell, 1990). 
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